Oct 30 18:33:25.446239 ip-10-0-133-80 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 18:33:25.841301 ip-10-0-133-80 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:33:25.841301 ip-10-0-133-80 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 18:33:25.841301 ip-10-0-133-80 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:33:25.841301 ip-10-0-133-80 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:33:25.845092 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845014    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:25.845092 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845091    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:25.845092 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845095    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:25.845186 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845098    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:25.845186 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845101    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:25.845186 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845104    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:25.845186 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845106    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:25.845186 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845109    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:25.845186 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845111    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:25.845186 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845114    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:25.845186 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845117    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:25.845186 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845120    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:25.845186 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845122    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:25.845186 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845125    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:25.845186 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845128    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:25.845186 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845131    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:25.845186 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845137    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:25.845186 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845140    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:25.845186 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845143    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:25.845186 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845145    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:25.845186 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845148    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:25.845186 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845151    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:25.845625 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845153    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:25.845625 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845156    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:25.845625 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845159    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:25.845625 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845162    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:25.845625 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845164    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:25.845625 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845167    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:25.845625 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845169    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:25.845625 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845172    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:25.845625 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845175    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:25.845625 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845177    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:25.845625 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845180    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:25.845625 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845182    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:25.845625 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845185    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:25.845625 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845187    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:25.845625 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845190    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:25.845625 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845193    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:25.845625 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845197    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:25.845625 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845199    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:25.845625 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845202    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:25.845625 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845205    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:25.846111 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845207    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:25.846111 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845210    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:25.846111 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845212    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:25.846111 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845215    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:25.846111 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845217    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:25.846111 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845220    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:25.846111 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845222    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:25.846111 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845226    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:25.846111 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845231    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:25.846111 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845234    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:25.846111 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845237    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:25.846111 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845239    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:25.846111 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845242    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:25.846111 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845246    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:25.846111 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845249    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:25.846111 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845251    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:25.846111 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845254    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:25.846111 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845257    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:25.846111 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845260    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:25.846111 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845262    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:25.846575 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845265    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:25.846575 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845267    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:25.846575 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845270    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:25.846575 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845272    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:25.846575 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845275    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:25.846575 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845279    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:25.846575 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845283    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:25.846575 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845286    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:25.846575 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845289    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:25.846575 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845292    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:25.846575 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845295    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:25.846575 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845297    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:25.846575 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845300    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:25.846575 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845302    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:25.846575 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845305    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:25.846575 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845307    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:25.846575 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845310    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:25.846575 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845313    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:25.846575 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845315    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:25.847063 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845318    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:25.847063 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845320    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:25.847063 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845323    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:25.847063 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845325    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:25.847063 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845329    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:25.847063 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845332    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:25.847063 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845335    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:25.847063 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845338    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:25.847063 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845341    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:25.847063 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845343    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:25.847063 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845346    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:25.847063 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845348    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:25.847063 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845351    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:25.847063 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845354    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:25.847063 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845356    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:25.847063 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845359    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:25.847063 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845362    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:25.847063 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845364    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:25.847063 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845367    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:25.847063 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845369    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:25.847541 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845372    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:25.847541 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845375    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:25.847541 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845378    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:25.847541 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845380    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:25.847541 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.845383    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:25.847541 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846161    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:25.847541 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846169    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:25.847541 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846173    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:25.847541 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846176    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:25.847541 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846179    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:25.847541 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846182    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:25.847541 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846185    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:25.847541 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846187    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:25.847541 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846190    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:25.847541 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846192    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:25.847541 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846195    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:25.847541 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846197    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:25.847541 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846200    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:25.847541 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846204    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:25.847541 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846207    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:25.848040 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846209    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:25.848040 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846212    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:25.848040 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846215    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:25.848040 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846218    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:25.848040 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846220    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:25.848040 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846223    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:25.848040 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846226    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:25.848040 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846228    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:25.848040 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846231    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:25.848040 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846233    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:25.848040 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846236    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:25.848040 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846238    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:25.848040 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846241    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:25.848040 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846243    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:25.848040 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846245    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:25.848040 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846248    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:25.848040 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846250    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:25.848040 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846254    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:25.848466 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846257    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:25.848466 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846259    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:25.848466 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846262    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:25.848466 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846264    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:25.848466 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846267    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:25.848466 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846269    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:25.848466 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846272    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:25.848466 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846274    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:25.848466 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846277    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:25.848466 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846279    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:25.848466 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846281    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:25.848466 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846284    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:25.848466 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846287    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:25.848466 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846290    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:25.848466 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846293    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:25.848466 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846295    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:25.848466 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846298    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:25.848466 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846300    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:25.848466 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846304    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:25.848466 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846308    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:25.848931 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846312    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:25.848931 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846316    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:25.848931 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846320    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:25.848931 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846323    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:25.848931 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846326    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:25.848931 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846328    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:25.848931 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846331    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:25.848931 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846334    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:25.848931 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846336    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:25.848931 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846339    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:25.848931 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846342    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:25.848931 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846344    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:25.848931 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846349    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:25.848931 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846370    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:25.848931 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846373    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:25.848931 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846377    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:25.848931 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846380    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:25.848931 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846383    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:25.848931 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846385    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:25.849401 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846388    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:25.849401 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846391    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:25.849401 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846394    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:25.849401 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846396    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:25.849401 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846399    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:25.849401 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846402    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:25.849401 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846405    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:25.849401 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846408    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:25.849401 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846410    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:25.849401 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846413    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:25.849401 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846416    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:25.849401 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846418    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:25.849401 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846421    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:25.849401 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846425    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:25.849401 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846427    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:25.849401 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846430    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:25.849401 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846432    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:25.849401 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846434    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:25.849401 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846437    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:25.849401 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846440    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:25.849871 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846442    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:25.849871 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846444    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:25.849871 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846447    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:25.849871 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846449    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:25.849871 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846452    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:25.849871 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846454    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:25.849871 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846457    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:25.849871 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846460    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:25.849871 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846462    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:25.849871 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846465    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:25.849871 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846467    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:25.849871 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846470    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:25.849871 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846472    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:25.849871 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.846478    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:25.849871 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847009    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:25.849871 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847015    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:25.849871 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847019    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:25.849871 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847022    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:25.849871 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847025    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:25.849871 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847027    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:25.850356 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847031    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:25.850356 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847033    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:25.850356 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847036    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:25.850356 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847039    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:25.850356 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847041    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:25.850356 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847044    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:25.850356 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847046    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:25.850356 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847049    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:25.850356 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847051    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:25.850356 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847054    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:25.850356 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847056    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:25.850356 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847059    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:25.850356 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847061    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:25.850356 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847064    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:25.850356 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847067    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:25.850356 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847070    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:25.850356 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847072    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:25.850356 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847075    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:25.850356 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847078    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:25.850356 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847081    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:25.850853 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847086    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:25.850853 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847090    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:25.850853 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847093    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:25.850853 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847096    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:25.850853 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847099    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:25.850853 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847103    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:25.850853 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847106    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:25.850853 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847109    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:25.850853 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847112    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:25.850853 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847114    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:25.850853 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847117    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:25.850853 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847120    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:25.850853 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847122    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:25.850853 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847125    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:25.850853 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847128    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:25.850853 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847131    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:25.850853 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847133    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:25.850853 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847136    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:25.850853 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847138    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:25.851346 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847141    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:25.851346 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847143    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:25.851346 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847146    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:25.851346 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847148    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:25.851346 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847151    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:25.851346 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847153    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:25.851346 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847156    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:25.851346 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847159    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:25.851346 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847163    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:25.851346 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847166    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:25.851346 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847170    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:25.851346 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847173    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:25.851346 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847176    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:25.851346 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847179    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:25.851346 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847190    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:25.851346 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847194    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:25.851346 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847196    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:25.851346 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847200    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:25.851346 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847204    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:25.851790 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847207    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:25.851790 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847209    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:25.851790 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847212    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:25.851790 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847215    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:25.851790 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847218    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:25.851790 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847220    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:25.851790 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847223    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:25.851790 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847225    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:25.851790 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847228    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:25.851790 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847231    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:25.851790 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847233    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:25.851790 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847236    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:25.851790 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847239    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:25.851790 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847241    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:25.851790 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847243    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:25.851790 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847246    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:25.851790 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847248    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:25.851790 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847251    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:25.851790 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847253    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:25.852289 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847256    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:25.852289 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847259    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:25.852289 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847261    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:25.852289 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847263    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:25.852289 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847266    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:25.852289 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847268    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:25.852289 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847271    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:25.852289 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847274    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:25.852289 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847276    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:25.852289 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847279    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:25.852289 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847281    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:25.852289 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847284    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:25.852289 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847288    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:25.852289 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847290    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:25.852289 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847293    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:25.852289 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847295    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:25.852289 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847298    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:25.852289 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847301    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:25.852289 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847303    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:25.852289 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847306    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847308    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847311    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.847313    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848497    2577 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848507    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848515    2577 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848520    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848524    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848528    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848532    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848536    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848539    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848542    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848546    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848549    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848552    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848555    2577 flags.go:64] FLAG: --cgroup-root=""
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848558    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848562    2577 flags.go:64] FLAG: --client-ca-file=""
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848564    2577 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848568    2577 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848597    2577 flags.go:64] FLAG: --cluster-domain=""
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848600    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 18:33:25.852757 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848604    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848608    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848611    2577 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848616    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848619    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848622    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848625    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848628    2577 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848631    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848634    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848637    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848640    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848644    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848647    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848650    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848653    2577 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848656    2577 flags.go:64] FLAG: --enable-server="true"
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848659    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848664    2577 flags.go:64] FLAG: --event-burst="100"
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848667    2577 flags.go:64] FLAG: --event-qps="50"
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848670    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848675    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848678    2577 flags.go:64] FLAG: --eviction-hard=""
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848682    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 18:33:25.853362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848685    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848688    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848691    2577 flags.go:64] FLAG: --eviction-soft=""
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848694    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848697    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848700    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848703    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848706    2577 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848709    2577 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848712    2577 flags.go:64] FLAG: --feature-gates=""
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848716    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848719    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848722    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848725    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848728    2577 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848731    2577 flags.go:64] FLAG: --help="false"
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848734    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848737    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848740    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848743    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848746    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848750    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848753    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 18:33:25.854071 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848756    2577 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848758    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848761    2577 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848764    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848767    2577 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848770    2577 flags.go:64] FLAG: --kube-reserved=""
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848774    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848777    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848780    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848783    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848785    2577 flags.go:64] FLAG: --lock-file=""
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848788    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848791    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848794    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848800    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848803    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848807    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848810    2577 flags.go:64] FLAG: --logging-format="text"
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848813    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848816    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848819    2577 flags.go:64] FLAG: --manifest-url=""
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848822    2577 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848826    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848829    2577 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848833    2577 flags.go:64] FLAG: --max-pods="110"
Oct 30 18:33:25.854624 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848836    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848839    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848842    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848844    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848847    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848850    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848853    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848860    2577 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848863    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848866    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848869    2577 flags.go:64] FLAG: --pod-cidr=""
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848872    2577 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848875    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848879    2577 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848882    2577 flags.go:64] FLAG: --port="10250"
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848885    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848888    2577 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-09e0ef491dbe1b3b7"
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848891    2577 flags.go:64] FLAG: --qos-reserved=""
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848894    2577 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848897    2577 flags.go:64] FLAG: --register-node="true"
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848900    2577 flags.go:64] FLAG: --register-with-taints=""
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848904    2577 flags.go:64] FLAG: --registry-burst="10"
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848906    2577 flags.go:64] FLAG: --registry-qps="5"
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848909    2577 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848912    2577 flags.go:64] FLAG: --reserved-memory=""
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848917    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 18:33:25.855242 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848920    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848923    2577 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848925    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848928    2577 flags.go:64] FLAG: --runonce="false"
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848932    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848935    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848938    2577 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848941    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848956    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848959    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848962    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848966    2577 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848969    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848972    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848975    2577 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848977    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848980    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848983    2577 flags.go:64] FLAG: --system-cgroups=""
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848986    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848992    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848995    2577 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.848998    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.849002    2577 flags.go:64] FLAG: --tls-min-version=""
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.849004    2577 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.849007    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 18:33:25.855861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.849010    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 18:33:25.856504 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.849013    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 18:33:25.856504 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.849016    2577 flags.go:64] FLAG: --v="2"
Oct 30 18:33:25.856504 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.849021    2577 flags.go:64] FLAG: --version="false"
Oct 30 18:33:25.856504 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.849025    2577 flags.go:64] FLAG: --vmodule=""
Oct 30 18:33:25.856504 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.849029    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 18:33:25.856504 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.849034    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 18:33:25.856504 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849172    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:25.856504 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849176    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:25.856504 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849179    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:25.856504 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849182    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:25.856504 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849185    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:25.856504 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849187    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:25.856504 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849190    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:25.856504 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849193    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:25.856504 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849195    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:25.856504 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849198    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:25.856504 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849201    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:25.856504 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849203    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:25.856504 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849206    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:25.856504 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849208    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:25.856504 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849211    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:25.857046 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849214    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:25.857046 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849216    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:25.857046 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849219    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:25.857046 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849222    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:25.857046 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849226    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:25.857046 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849228    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:25.857046 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849231    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:25.857046 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849233    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:25.857046 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849236    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:25.857046 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849238    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:25.857046 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849241    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:25.857046 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849243    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:25.857046 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849246    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:25.857046 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849248    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:25.857046 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849251    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:25.857046 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849253    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:25.857046 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849257    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:25.857046 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849267    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:25.857046 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849270    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:25.857046 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849272    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:25.857521 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849275    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:25.857521 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849278    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:25.857521 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849280    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:25.857521 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849283    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:25.857521 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849286    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:25.857521 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849289    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:25.857521 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849291    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:25.857521 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849294    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:25.857521 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849296    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:25.857521 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849299    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:25.857521 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849301    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:25.857521 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849304    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:25.857521 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849306    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:25.857521 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849309    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:25.857521 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849311    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:25.857521 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849314    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:25.857521 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849317    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:25.857521 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849320    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:25.857521 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849323    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:25.857521 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849325    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:25.858007 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849328    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:25.858007 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849330    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:25.858007 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849333    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:25.858007 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849335    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:25.858007 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849338    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:25.858007 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849340    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:25.858007 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849342    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:25.858007 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849345    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:25.858007 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849349    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:25.858007 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849351    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:25.858007 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849354    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:25.858007 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849357    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:25.858007 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849361    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:25.858007 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849365    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:25.858007 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849368    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:25.858007 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849370    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:25.858007 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849373    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:25.858007 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849375    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:25.858007 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849378    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:25.858460 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849380    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:25.858460 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849383    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:25.858460 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849385    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:25.858460 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849388    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:25.858460 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849390    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:25.858460 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849393    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:25.858460 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849395    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:25.858460 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849398    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:25.858460 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849400    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:25.858460 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849404    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:25.858460 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849407    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:25.858460 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849410    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:25.858460 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849413    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:25.858460 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849415    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:25.858460 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849419    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:25.858460 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849423    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:25.858460 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849426    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:25.858460 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849429    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:25.858460 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849431    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:25.858919 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849434    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:25.858919 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849436    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:25.858919 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849440    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:25.858919 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849443    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:25.858919 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849446    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:25.858919 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849449    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:25.858919 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849451    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:25.858919 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849454    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:25.858919 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849456    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:25.858919 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849459    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:25.858919 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849461    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:25.858919 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849464    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:25.858919 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849466    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:25.858919 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.849493    2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:33:25.858919 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849534    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:25.858919 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849539    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:25.859318 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849542    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:25.859318 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849545    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:25.859318 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849547    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:25.859318 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849550    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:25.859318 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849552    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:25.859318 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849555    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:25.859318 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849559    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:25.859318 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849562    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:25.859318 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849564    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:25.859318 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849567    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:25.859318 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849569    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:25.859318 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849572    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:25.859318 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849575    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:25.859318 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849577    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:25.859318 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849580    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:25.859318 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849582    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:25.859318 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849585    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:25.859318 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849587    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:25.859318 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849590    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:25.859318 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849593    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:25.859804 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849610    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:25.859804 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849615    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:25.859804 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849620    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:25.859804 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849624    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:25.859804 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849628    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:25.859804 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849631    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:25.859804 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849634    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:25.859804 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849637    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:25.859804 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849639    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:25.859804 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849642    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:25.859804 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849645    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:25.859804 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849647    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:25.859804 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849650    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:25.859804 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849652    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:25.859804 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849655    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:25.859804 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849657    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:25.859804 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849660    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:25.859804 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849662    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:25.859804 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849665    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:25.860269 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849668    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:25.860269 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849670    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:25.860269 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849673    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:25.860269 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849675    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:25.860269 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849678    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:25.860269 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849680    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:25.860269 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849682    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:25.860269 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849685    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:25.860269 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849687    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:25.860269 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849690    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:25.860269 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849692    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:25.860269 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849695    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:25.860269 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849698    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:25.860269 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849701    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:25.860269 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849703    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:25.860269 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849706    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:25.860269 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849709    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:25.860269 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849712    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:25.860269 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849714    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:25.860269 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849717    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:25.860738 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849720    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:25.860738 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849722    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:25.860738 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849725    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:25.860738 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849728    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:25.860738 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849730    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:25.860738 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849734    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:25.860738 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849738    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:25.860738 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849741    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:25.860738 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849744    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:25.860738 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849746    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:25.860738 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849749    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:25.860738 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849751    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:25.860738 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849754    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:25.860738 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849756    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:25.860738 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849759    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:25.860738 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849761    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:25.860738 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849763    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:25.860738 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849766    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:25.860738 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849768    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:25.861197 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849771    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:25.861197 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849773    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:25.861197 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849776    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:25.861197 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849778    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:25.861197 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849781    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:25.861197 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849785    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:25.861197 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849788    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:25.861197 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849792    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:25.861197 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849795    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:25.861197 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849798    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:25.861197 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849801    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:25.861197 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849804    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:25.861197 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849807    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:25.861197 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849809    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:25.861197 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849812    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:25.861197 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849815    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:25.861197 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849818    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:25.861197 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849820    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:25.861197 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849823    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:25.861642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849825    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:25.861642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849828    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:25.861642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849830    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:25.861642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849833    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:25.861642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849835    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:25.861642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849838    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:25.861642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.849840    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:25.861642 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.856550    2577 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 18:33:25.861642 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.856565    2577 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 18:33:25.861642 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.856581    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 18:33:25.861642 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.856586    2577 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 18:33:25.861642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856661    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:25.861642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856666    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:25.861642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856669    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:25.861642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856672    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:25.861642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856675    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:25.861642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856679    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:25.861642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856682    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:25.861642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856684    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:25.861642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856687    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:25.862119 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856689    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:25.862119 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856692    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:25.862119 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856695    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:25.862119 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856705    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:25.862119 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856708    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:25.862119 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856711    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:25.862119 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856713    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:25.862119 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856716    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:25.862119 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856718    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:25.862119 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856721    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:25.862119 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856724    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:25.862119 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856726    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:25.862119 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856729    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:25.862119 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856731    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:25.862119 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856734    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:25.862119 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856736    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:25.862119 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856739    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:25.862119 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856741    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:25.862119 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856744    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:25.862119 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856746    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:25.862620 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856750    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:25.862620 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856753    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:25.862620 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856755    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:25.862620 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856758    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:25.862620 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856761    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:25.862620 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856763    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:25.862620 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856766    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:25.862620 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856769    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:25.862620 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856771    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:25.862620 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856774    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:25.862620 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856776    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:25.862620 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856778    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:25.862620 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856781    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:25.862620 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856784    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:25.862620 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856786    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:25.862620 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856789    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:25.862620 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856799    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:25.862620 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856802    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:25.862620 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856806    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:25.863244 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856809    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:25.863244 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856812    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:25.863244 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856815    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:25.863244 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856817    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:25.863244 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856820    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:25.863244 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856822    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:25.863244 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856825    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:25.863244 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856827    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:25.863244 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856830    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:25.863244 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856832    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:25.863244 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856835    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:25.863244 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856837    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:25.863244 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856839    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:25.863244 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856843    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:25.863244 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856845    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:25.863244 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856848    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:25.863244 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856850    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:25.863244 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856853    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:25.863244 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856855    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:25.864017 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856858    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:25.864017 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856860    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:25.864017 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856863    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:25.864017 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856865    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:25.864017 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856868    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:25.864017 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856870    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:25.864017 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856873    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:25.864017 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856876    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:25.864017 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856879    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:25.864017 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856883    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:25.864017 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856886    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:25.864017 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856899    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:25.864017 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856902    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:25.864017 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856905    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:25.864017 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856908    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:25.864017 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856910    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:25.864017 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856913    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:25.864017 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856915    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:25.864017 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856918    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:25.864639 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856920    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:25.864639 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856923    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:25.864639 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856925    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:25.864639 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856928    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:25.864639 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856930    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:25.864639 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856933    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:25.864639 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856935    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:25.864639 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856938    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:25.864639 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856940    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:25.864639 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856943    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:25.864639 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856962    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:25.864639 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856965    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:25.864639 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856967    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:25.864639 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856970    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:25.864639 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856972    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:25.864639 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856975    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:25.864639 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856977    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:25.864639 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856980    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:25.864639 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856982    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:25.864639 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.856984    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:25.865303 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.857012    2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:33:25.865303 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857204    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:25.865303 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857210    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:25.865303 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857213    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:25.865303 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857217    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:25.865303 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857225    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:25.865303 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857228    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:25.865303 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857230    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:25.865303 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857233    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:25.865303 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857235    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:25.865303 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857238    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:25.865303 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857241    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:25.865303 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857243    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:25.865303 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857245    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:25.865303 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857248    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:25.865303 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857250    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:25.865739 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857253    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:25.865739 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857255    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:25.865739 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857258    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:25.865739 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857260    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:25.865739 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857264    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:25.865739 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857268    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:25.865739 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857271    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:25.865739 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857273    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:25.865739 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857276    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:25.865739 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857279    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:25.865739 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857281    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:25.865739 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857284    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:25.865739 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857286    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:25.865739 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857289    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:25.865739 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857291    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:25.865739 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857294    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:25.865739 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857296    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:25.865739 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857299    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:25.865739 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857301    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:25.866229 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857304    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:25.866229 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857306    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:25.866229 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857309    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:25.866229 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857316    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:25.866229 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857319    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:25.866229 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857322    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:25.866229 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857324    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:25.866229 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857326    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:25.866229 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857329    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:25.866229 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857331    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:25.866229 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857333    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:25.866229 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857336    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:25.866229 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857339    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:25.866229 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857341    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:25.866229 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857343    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:25.866229 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857346    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:25.866229 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857348    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:25.866229 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857351    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:25.866229 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857353    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:25.866229 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857356    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:25.866760 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857358    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:25.866760 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857361    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:25.866760 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857363    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:25.866760 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857366    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:25.866760 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857368    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:25.866760 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857371    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:25.866760 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857373    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:25.866760 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857375    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:25.866760 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857378    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:25.866760 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857380    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:25.866760 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857382    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:25.866760 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857386    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:25.866760 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857389    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:25.866760 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857391    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:25.866760 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857394    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:25.866760 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857396    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:25.866760 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857404    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:25.866760 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857407    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:25.866760 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857409    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:25.867316 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857412    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:25.867316 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857414    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:25.867316 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857416    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:25.867316 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857419    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:25.867316 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857421    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:25.867316 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857424    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:25.867316 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857426    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:25.867316 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857429    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:25.867316 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857431    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:25.867316 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857434    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:25.867316 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857436    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:25.867316 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857439    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:25.867316 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857441    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:25.867316 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857444    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:25.867316 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857446    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:25.867316 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857452    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:25.867316 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857455    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:25.867316 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857457    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:25.867316 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857460    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:25.867766 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857462    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:25.867766 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857464    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:25.867766 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857467    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:25.867766 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857469    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:25.867766 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857471    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:25.867766 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857474    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:25.867766 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857478    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:25.867766 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857481    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:25.867766 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857483    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:25.867766 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857486    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:25.867766 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857488    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:25.867766 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857496    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:25.867766 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857499    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:25.867766 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857501    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:25.867766 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.857524    2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:33:25.867766 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857570    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:25.868165 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857574    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:25.868165 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857578    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:25.868165 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857580    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:25.868165 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857583    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:25.868165 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857586    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:25.868165 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857589    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:25.868165 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857591    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:25.868165 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857593    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:25.868165 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857596    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:25.868165 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857598    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:25.868165 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857601    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:25.868165 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857603    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:25.868165 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857606    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:25.868165 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857608    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:25.868165 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857610    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:25.868165 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857613    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:25.868165 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857616    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:25.868165 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857618    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:25.868165 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857620    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:25.868165 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857623    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:25.868642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857625    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:25.868642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857627    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:25.868642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857631    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:25.868642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857634    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:25.868642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857636    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:25.868642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857638    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:25.868642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857641    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:25.868642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857644    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:25.868642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857651    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:25.868642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857654    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:25.868642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857656    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:25.868642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857659    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:25.868642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857661    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:25.868642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857664    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:25.868642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857666    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:25.868642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857669    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:25.868642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857673    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:25.868642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857676    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:25.868642 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857680    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:25.869118 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857682    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:25.869118 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857685    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:25.869118 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857688    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:25.869118 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857690    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:25.869118 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857693    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:25.869118 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857695    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:25.869118 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857697    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:25.869118 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857700    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:25.869118 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857702    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:25.869118 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857705    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:25.869118 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857707    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:25.869118 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857709    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:25.869118 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857712    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:25.869118 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857714    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:25.869118 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857717    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:25.869118 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857720    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:25.869118 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857723    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:25.869118 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857726    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:25.869118 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857728    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:25.869559 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857731    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:25.869559 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857734    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:25.869559 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857736    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:25.869559 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857744    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:25.869559 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857747    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:25.869559 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857749    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:25.869559 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857751    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:25.869559 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857754    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:25.869559 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857756    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:25.869559 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857758    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:25.869559 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857761    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:25.869559 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857763    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:25.869559 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857766    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:25.869559 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857768    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:25.869559 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857770    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:25.869559 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857773    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:25.869559 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857775    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:25.869559 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857778    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:25.869559 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857780    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:25.869559 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857782    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:25.870078 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857785    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:25.870078 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857787    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:25.870078 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857789    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:25.870078 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857792    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:25.870078 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857794    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:25.870078 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857797    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:25.870078 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857799    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:25.870078 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857801    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:25.870078 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857805    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:25.870078 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857813    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:25.870078 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857816    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:25.870078 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857818    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:25.870078 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857820    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:25.870078 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857823    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:25.870078 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857825    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:25.870078 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857827    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:25.870078 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857835    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:25.870078 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857838    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:25.870078 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857842    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:25.870576 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857845    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:25.870576 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857848    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:25.870576 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857851    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:25.870576 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857853    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:25.870576 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857856    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:25.870576 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857858    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:25.870576 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857861    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:25.870576 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:25.857863    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:25.870576 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.857911    2577 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 18:33:25.870576 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.859793    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 18:33:25.870576 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.860698    2577 server.go:1014] "Starting client certificate rotation"
Oct 30 18:33:25.870576 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.860804    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 18:33:25.870576 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.860844    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 18:33:25.870576 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.864840    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:33:25.870576 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.866551    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:33:25.878004 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.877986    2577 log.go:25] "Validated CRI v1 runtime API"
Oct 30 18:33:25.880819 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.880804    2577 log.go:25] "Validated CRI v1 image API"
Oct 30 18:33:25.881647 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.881623    2577 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 18:33:25.884152 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.884124    2577 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b5bc3841-233d-4f6b-9a64-6dcb4c75c1c8:/dev/nvme0n1p3 f3278fdb-3dec-4bc4-9455-90812e3d7be6:/dev/nvme0n1p4]
Oct 30 18:33:25.884213 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.884152    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 18:33:25.890055 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.889928    2577 manager.go:217] Machine: {Timestamp:2026-10-30 18:33:25.8886766 +0000 UTC m=+0.324940721 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3088826 MemoryCapacity:32812007424 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2250145727e056726470fd8e9b60fa SystemUUID:ec225014-5727-e056-7264-70fd8e9b60fa BootID:386c8f48-c81a-4b2c-8186-4f5197fd36e0 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406003712 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406003712 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2f:6d:c8:fb:65 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2f:6d:c8:fb:65 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:55:95:4e:c7:e6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812007424 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 18:33:25.890055 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.890053    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 18:33:25.890169 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.890128    2577 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 18:33:25.891789 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.891762    2577 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 18:33:25.891918 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.891792    2577 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-80.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 18:33:25.891976 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.891922    2577 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 18:33:25.891976 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.891930    2577 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 18:33:25.892037 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.891976    2577 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:33:25.892037 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.891995    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:33:25.892661 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.892652    2577 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 18:33:25.896257 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.896245    2577 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 18:33:25.896379 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.896370    2577 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 18:33:25.896976 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.896965    2577 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 18:33:25.897013 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.896981    2577 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 18:33:25.897013 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.896991    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 18:33:25.897013 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.897000    2577 kubelet.go:405] "Adding apiserver pod source"
Oct 30 18:33:25.897013 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.897008    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 18:33:25.897380 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.897369    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:33:25.899327 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.899312    2577 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 18:33:25.903413 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.903401    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:33:25.905223 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.904872    2577 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 18:33:25.905223 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.904921    2577 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 18:33:25.905223 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.905137    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 18:33:25.905223 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.905155    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 18:33:25.905223 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.905166    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 18:33:25.905223 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.905175    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 18:33:25.905223 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.905191    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 18:33:25.905223 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.905201    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 18:33:25.905663 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.905650    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 18:33:25.905730 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.905721    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 18:33:25.905792 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.905785    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 18:33:25.905848 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.905841    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 18:33:25.905910 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.905902    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 18:33:25.906006 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.905986    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 18:33:25.906732 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.906721    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 18:33:25.906788 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.906744    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 18:33:25.910499 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.910483    2577 server.go:1263] "Started kubelet"
Oct 30 18:33:25.910590 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.910558    2577 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 18:33:25.910636 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.910573    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 18:33:25.910675 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.910656    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 18:33:25.911205 ip-10-0-133-80 systemd[1]: Started Kubernetes Kubelet.
Oct 30 18:33:25.911637 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.911622    2577 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 18:33:25.912854 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.912833    2577 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 18:33:25.916820 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:25.916801    2577 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 18:33:25.917291 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.917276    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 18:33:25.917704 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.917689    2577 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 18:33:25.917879 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:25.917855    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-80.us-west-1.compute.internal\" not found"
Oct 30 18:33:25.918375 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.918356    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 18:33:25.918375 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.918359    2577 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 18:33:25.918521 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.918388    2577 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 18:33:25.918521 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.918453    2577 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 18:33:25.918521 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.918463    2577 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 18:33:25.918521 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.918483    2577 factory.go:55] Registering systemd factory
Oct 30 18:33:25.918521 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.918499    2577 factory.go:223] Registration of the systemd container factory successfully
Oct 30 18:33:25.918739 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.918716    2577 factory.go:153] Registering CRI-O factory
Oct 30 18:33:25.918739 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.918729    2577 factory.go:223] Registration of the crio container factory successfully
Oct 30 18:33:25.918790 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.918768    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 18:33:25.918790 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.918783    2577 factory.go:103] Registering Raw factory
Oct 30 18:33:25.918845 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.918795    2577 manager.go:1199] Started watching for new ooms in manager
Oct 30 18:33:25.919614 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.919596    2577 manager.go:319] Starting recovery of all containers
Oct 30 18:33:25.930677 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.930657    2577 manager.go:324] Recovery completed
Oct 30 18:33:25.935222 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.935207    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:25.936748 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.936733    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:25.936817 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.936761    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:25.936817 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.936775    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:25.937785 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.937768    2577 cpu_manager.go:226] "Starting" policy="none"
Oct 30 18:33:25.937785 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.937783    2577 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 18:33:25.937887 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.937797    2577 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 18:33:25.937887 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.937776    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:25.938451 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.938437    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:25.938517 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.938467    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:25.938517 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.938482    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:25.940002 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.939987    2577 policy_none.go:50] "Start"
Oct 30 18:33:25.940067 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.940008    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 18:33:25.940067 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.940025    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 18:33:25.942119 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.942107    2577 policy_none.go:44] "Start"
Oct 30 18:33:25.977619 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.977495    2577 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 18:33:25.981414 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.981399    2577 manager.go:341] "Starting Device Plugin manager"
Oct 30 18:33:25.981478 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:25.981428    2577 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 18:33:25.981478 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.981437    2577 server.go:87] "Starting device plugin registration server"
Oct 30 18:33:25.981640 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.981630    2577 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 18:33:25.981673 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.981654    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 18:33:25.981763 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.981745    2577 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 18:33:25.981842 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.981831    2577 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 18:33:25.981842 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:25.981839    2577 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 18:33:25.982348 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:25.982331    2577 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 18:33:25.982437 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:25.982372    2577 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-80.us-west-1.compute.internal\" not found"
Oct 30 18:33:26.044049 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.044014    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 18:33:26.046051 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.045266    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 18:33:26.046051 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.045286    2577 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 18:33:26.046051 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.045307    2577 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 18:33:26.046051 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:26.045345    2577 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 18:33:26.055296 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:26.055268    2577 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-80.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 18:33:26.056222 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:26.055266    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-80.us-west-1.compute.internal.18e36257cfbbc3a7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-80.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-80.us-west-1.compute.internal,},FirstTimestamp:2026-10-30 18:33:25.910455207 +0000 UTC m=+0.346719328,LastTimestamp:2026-10-30 18:33:25.910455207 +0000 UTC m=+0.346719328,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-80.us-west-1.compute.internal,}"
Oct 30 18:33:26.081753 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.081727    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:26.083399 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.083386    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:26.083460 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.083414    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:26.083460 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.083424    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:26.083460 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.083447    2577 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:33:26.103751 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.103701    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-80.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 18:33:26.146094 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.146066    2577 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal"]
Oct 30 18:33:26.146189 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.146122    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:26.147021 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.147000    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:26.147140 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.147031    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:26.147140 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.147046    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:26.149216 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.149205    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:26.149376 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.149361    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:33:26.149409 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.149392    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:26.150033 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.150015    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:26.150113 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.150017    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:26.150113 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.150069    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:26.150113 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.150079    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:26.150113 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.150050    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:26.150230 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.150132    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:26.152092 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.152080    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:33:26.152135 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.152104    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:26.152835 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.152823    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:26.152884 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.152846    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:26.152884 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.152855    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:26.156784 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.156761    2577 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 18:33:26.156849 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.156800    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lwq2j"
Oct 30 18:33:26.182577 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:26.182559    2577 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-80.us-west-1.compute.internal\" not found" node="ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:33:26.185101 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:26.185085    2577 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-80.us-west-1.compute.internal\" not found" node="ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:33:26.215124 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.215105    2577 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:33:26.216794 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.216779    2577 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:33:26.220625 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.220610    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d085b0a4a8eed9a1bd72245b7d7a6e5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal\" (UID: \"d085b0a4a8eed9a1bd72245b7d7a6e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:33:26.220696 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.220632    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d085b0a4a8eed9a1bd72245b7d7a6e5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal\" (UID: \"d085b0a4a8eed9a1bd72245b7d7a6e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:33:26.220696 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.220647    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ff8defe0c0f1527a0efd6540e2b0bdc3-config\") pod \"kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal\" (UID: \"ff8defe0c0f1527a0efd6540e2b0bdc3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:33:26.314111 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.314086    2577 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:33:26.316375 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.316353    2577 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:33:26.317264 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.317188    2577 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:33:26.317909 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.317887    2577 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:33:26.321623 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.321603    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d085b0a4a8eed9a1bd72245b7d7a6e5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal\" (UID: \"d085b0a4a8eed9a1bd72245b7d7a6e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:33:26.321713 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.321629    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d085b0a4a8eed9a1bd72245b7d7a6e5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal\" (UID: \"d085b0a4a8eed9a1bd72245b7d7a6e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:33:26.321713 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.321646    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ff8defe0c0f1527a0efd6540e2b0bdc3-config\") pod \"kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal\" (UID: \"ff8defe0c0f1527a0efd6540e2b0bdc3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:33:26.321713 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.321690    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ff8defe0c0f1527a0efd6540e2b0bdc3-config\") pod \"kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal\" (UID: \"ff8defe0c0f1527a0efd6540e2b0bdc3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:33:26.321831 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.321718    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d085b0a4a8eed9a1bd72245b7d7a6e5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal\" (UID: \"d085b0a4a8eed9a1bd72245b7d7a6e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:33:26.321831 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.321720    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d085b0a4a8eed9a1bd72245b7d7a6e5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal\" (UID: \"d085b0a4a8eed9a1bd72245b7d7a6e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:33:26.382881 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.382822    2577 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:33:26.383012 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.382939    2577 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:33:26.446474 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.446451    2577 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:33:26.484993 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.484968    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:33:26.487850 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.487834    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:33:26.860747 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.860669    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 18:33:26.861256 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.860804    2577 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 18:33:26.861256 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.860844    2577 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 18:33:26.861256 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.860857    2577 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 18:33:26.861256 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:26.860905    2577 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a4d303792d30e478da0cee90b09d474b-fa61d523e62daa65.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-133-80.us-west-1.compute.internal.18e36257d14d3d22\": read tcp 10.0.133.80:33472->77.114.47.6:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-133-80.us-west-1.compute.internal.18e36257d14d3d22  default   5089 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-80.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-80.us-west-1.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-80.us-west-1.compute.internal,},FirstTimestamp:2026-10-30 18:33:25 +0000 UTC,LastTimestamp:2026-10-30 18:33:26.083418401 +0000 UTC m=+0.519682522,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-80.us-west-1.compute.internal,}"
Oct 30 18:33:26.898065 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.898044    2577 apiserver.go:52] "Watching apiserver"
Oct 30 18:33:26.918173 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:26.918156    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 18:33:27.161088 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.160996    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:28:26 +0000 UTC" deadline="2028-04-16 20:52:17.808937685 +0000 UTC"
Oct 30 18:33:27.161088 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.161037    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12818h18m50.647902552s"
Oct 30 18:33:27.178577 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.178556    2577 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 18:33:27.178898 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.178880    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-tzdf5","openshift-network-diagnostics/network-check-target-5gvjh","openshift-ovn-kubernetes/ovnkube-node-vq8lg","kube-system/konnectivity-agent-j4w9c","kube-system/kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5","openshift-cluster-node-tuning-operator/tuned-nnld2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal","openshift-multus/network-metrics-daemon-h5nm6","kube-system/global-pull-secret-syncer-gfs79","openshift-dns/node-resolver-nr7xl","openshift-multus/multus-additional-cni-plugins-8jpvz","openshift-network-operator/iptables-alerter-dvxpz","openshift-image-registry/node-ca-8kldf"]
Oct 30 18:33:27.181390 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.181372    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.183436 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.183417    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:27.183510 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:27.183484    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:33:27.185569 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.185555    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.187584 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.187570    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j4w9c"
Oct 30 18:33:27.189677 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.189663    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.191715 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.191698    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.193782 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.193764    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:27.193865 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:27.193821    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:33:27.195973 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.195942    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:27.196057 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:27.196011    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:33:27.198280 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.198265    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nr7xl"
Oct 30 18:33:27.201300 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.201277    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.204019 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.204002    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dvxpz"
Oct 30 18:33:27.206086 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.206072    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8kldf"
Oct 30 18:33:27.219139 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.219118    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 18:33:27.226448 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226431    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-sys-fs\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.226520 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226456    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-slash\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.226520 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226472    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-registration-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.226520 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226488    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7bfc481b-9824-43c5-acf0-436641114b03-serviceca\") pod \"node-ca-8kldf\" (UID: \"7bfc481b-9824-43c5-acf0-436641114b03\") " pod="openshift-image-registry/node-ca-8kldf"
Oct 30 18:33:27.226520 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226506    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-conf-dir\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.226678 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226552    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-systemd-units\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.226678 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226583    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-cni-netd\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.226678 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226609    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99psk\" (UniqueName: \"kubernetes.io/projected/d3fa116f-e372-4a36-9a57-4832dc1440b3-kube-api-access-99psk\") pod \"node-resolver-nr7xl\" (UID: \"d3fa116f-e372-4a36-9a57-4832dc1440b3\") " pod="openshift-dns/node-resolver-nr7xl"
Oct 30 18:33:27.226678 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226625    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.226678 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226640    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-etc-kubernetes\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.226897 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226678    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-var-lib-openvswitch\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.226897 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226722    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-etc-openvswitch\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.226897 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226746    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-var-lib-kubelet\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.226897 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226766    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3fa116f-e372-4a36-9a57-4832dc1440b3-tmp-dir\") pod \"node-resolver-nr7xl\" (UID: \"d3fa116f-e372-4a36-9a57-4832dc1440b3\") " pod="openshift-dns/node-resolver-nr7xl"
Oct 30 18:33:27.226897 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226781    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-cni-binary-copy\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.226897 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226797    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0c816c10-1c32-4538-8724-834ebb405960-iptables-alerter-script\") pod \"iptables-alerter-dvxpz\" (UID: \"0c816c10-1c32-4538-8724-834ebb405960\") " pod="openshift-network-operator/iptables-alerter-dvxpz"
Oct 30 18:33:27.226897 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226811    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-cni-binary-copy\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.226897 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226824    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-daemon-config\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.226897 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226838    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-cni-bin\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.226897 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-socket-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.226897 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226886    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-run\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.226897 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226900    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-sys\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.227383 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226923    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:27.227383 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.226961    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-run-openvswitch\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.227383 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227002    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-var-lib-cni-multus\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.227383 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227039    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-run-multus-certs\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.227383 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227077    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-etc-selinux\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.227383 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227097    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-tmp\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.227383 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227111    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c816c10-1c32-4538-8724-834ebb405960-host-slash\") pod \"iptables-alerter-dvxpz\" (UID: \"0c816c10-1c32-4538-8724-834ebb405960\") " pod="openshift-network-operator/iptables-alerter-dvxpz"
Oct 30 18:33:27.227383 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227129    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-var-lib-kubelet\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.227383 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227152    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-run-ovn\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.227383 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227168    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b1735618-e0f6-49b1-ade6-8fef300791a1-ovnkube-script-lib\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.227383 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227203    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zzhj\" (UniqueName: \"kubernetes.io/projected/b1735618-e0f6-49b1-ade6-8fef300791a1-kube-api-access-2zzhj\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.227383 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227239    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtpk7\" (UniqueName: \"kubernetes.io/projected/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-kube-api-access-vtpk7\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.227383 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227273    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl92s\" (UniqueName: \"kubernetes.io/projected/0c816c10-1c32-4538-8724-834ebb405960-kube-api-access-zl92s\") pod \"iptables-alerter-dvxpz\" (UID: \"0c816c10-1c32-4538-8724-834ebb405960\") " pod="openshift-network-operator/iptables-alerter-dvxpz"
Oct 30 18:33:27.227383 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227306    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-system-cni-dir\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.227383 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227333    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-run-k8s-cni-cncf-io\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.227383 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227357    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-run-systemd\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.228029 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227383    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-sysconfig\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.228029 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227409    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-lib-modules\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.228029 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227436    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-run-netns\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.228029 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227458    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-kubernetes\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.228029 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227481    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-tuned\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.228029 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227505    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.228029 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227530    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-hostroot\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.228029 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227553    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-var-lib-cni-bin\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.228029 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227580    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-run-netns\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.228029 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227601    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-log-socket\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.228029 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227624    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-device-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.228029 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227658    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-cni-dir\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.228029 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227674    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-modprobe-d\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.228029 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227687    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25k6c\" (UniqueName: \"kubernetes.io/projected/08a8e709-63af-4d71-b084-cf533ea620e5-kube-api-access-25k6c\") pod \"network-metrics-daemon-h5nm6\" (UID: \"08a8e709-63af-4d71-b084-cf533ea620e5\") " pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:27.228029 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227702    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-socket-dir-parent\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.228029 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227717    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-kubelet\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.228619 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227731    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b1735618-e0f6-49b1-ade6-8fef300791a1-ovnkube-config\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.228619 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227746    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.228619 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227760    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d3fa116f-e372-4a36-9a57-4832dc1440b3-hosts-file\") pod \"node-resolver-nr7xl\" (UID: \"d3fa116f-e372-4a36-9a57-4832dc1440b3\") " pod="openshift-dns/node-resolver-nr7xl"
Oct 30 18:33:27.228619 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227775    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.228619 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227804    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-run-ovn-kubernetes\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.228619 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227850    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b1735618-e0f6-49b1-ade6-8fef300791a1-env-overrides\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.228619 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227871    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.228619 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227887    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px2j7\" (UniqueName: \"kubernetes.io/projected/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-kube-api-access-px2j7\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.228619 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227903    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-sysctl-d\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.228619 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227916    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-systemd\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.228619 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227964    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-host\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.228619 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227981    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b1735618-e0f6-49b1-ade6-8fef300791a1-ovn-node-metrics-cert\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.228619 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.227994    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3443d97d-774b-4df3-abe3-3ca2d6b574b6-agent-certs\") pod \"konnectivity-agent-j4w9c\" (UID: \"3443d97d-774b-4df3-abe3-3ca2d6b574b6\") " pod="kube-system/konnectivity-agent-j4w9c"
Oct 30 18:33:27.228619 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.228009    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs\") pod \"network-metrics-daemon-h5nm6\" (UID: \"08a8e709-63af-4d71-b084-cf533ea620e5\") " pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:27.228619 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.228023    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6300215e-2302-47ed-bf47-05a165277c08-kubelet-config\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:27.228619 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.228055    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6300215e-2302-47ed-bf47-05a165277c08-dbus\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:27.229085 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.228089    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdl78\" (UniqueName: \"kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78\") pod \"network-check-target-5gvjh\" (UID: \"a864943c-d7eb-4486-99a6-7961573b57d4\") " pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:27.229085 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.228116    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-system-cni-dir\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.229085 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.228143    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-cnibin\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.229085 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.228166    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm96j\" (UniqueName: \"kubernetes.io/projected/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-kube-api-access-cm96j\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.229085 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.228197    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-sysctl-conf\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.229085 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.228225    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7bfc481b-9824-43c5-acf0-436641114b03-host\") pod \"node-ca-8kldf\" (UID: \"7bfc481b-9824-43c5-acf0-436641114b03\") " pod="openshift-image-registry/node-ca-8kldf"
Oct 30 18:33:27.229085 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.228249    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g5sn\" (UniqueName: \"kubernetes.io/projected/7bfc481b-9824-43c5-acf0-436641114b03-kube-api-access-5g5sn\") pod \"node-ca-8kldf\" (UID: \"7bfc481b-9824-43c5-acf0-436641114b03\") " pod="openshift-image-registry/node-ca-8kldf"
Oct 30 18:33:27.229085 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.228278    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-node-log\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.229085 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.228305    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3443d97d-774b-4df3-abe3-3ca2d6b574b6-konnectivity-ca\") pod \"konnectivity-agent-j4w9c\" (UID: \"3443d97d-774b-4df3-abe3-3ca2d6b574b6\") " pod="kube-system/konnectivity-agent-j4w9c"
Oct 30 18:33:27.229085 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.228338    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-cnibin\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.229085 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.228365    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms9kr\" (UniqueName: \"kubernetes.io/projected/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-kube-api-access-ms9kr\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.237001 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.236978    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 18:33:27.237169 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.237150    2577 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 18:33:27.237221 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.237212    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gl8p6"
Oct 30 18:33:27.237659 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.237639    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 18:33:27.237659 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.237643    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 18:33:27.237805 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.237684    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-b7gnv\""
Oct 30 18:33:27.237805 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.237717    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 18:33:27.238223 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.238208    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 18:33:27.239796 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.239776    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 18:33:27.239796 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.239790    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 18:33:27.239930 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.239860    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 18:33:27.240069 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.240055    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 18:33:27.240146 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.240133    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jkxcr\""
Oct 30 18:33:27.240202 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.240172    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 18:33:27.243899 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.243885    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 18:33:27.243986 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.243973    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-g6mvb\""
Oct 30 18:33:27.244060 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.244048    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 18:33:27.244259 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.244246    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 18:33:27.244299 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.244262    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4bpx7\""
Oct 30 18:33:27.285876 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.285858    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 18:33:27.285876 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.285868    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 18:33:27.286058 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.285869    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 18:33:27.286929 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.286914    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5qqcs\""
Oct 30 18:33:27.328747 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.328725    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-cni-binary-copy\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.328838 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.328753    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-daemon-config\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.328838 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.328770    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-cni-bin\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.328838 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.328783    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-socket-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.328838 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.328797    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-run\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.328987 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.328848    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-cni-bin\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.328987 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.328881    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-socket-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.328987 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.328892    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-sys\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.328987 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.328919    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:27.328987 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.328941    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-run-openvswitch\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.328987 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.328985    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-var-lib-cni-multus\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.329244 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329007    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-run-multus-certs\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.329244 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329009    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-run\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.329244 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329008    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-sys\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.329244 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329029    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-etc-selinux\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.329244 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329039    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-run-openvswitch\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.329244 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329057    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-var-lib-cni-multus\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.329244 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329057    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-run-multus-certs\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.329244 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329053    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-tmp\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.329244 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:27.329094    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:27.329244 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329109    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c816c10-1c32-4538-8724-834ebb405960-host-slash\") pod \"iptables-alerter-dvxpz\" (UID: \"0c816c10-1c32-4538-8724-834ebb405960\") " pod="openshift-network-operator/iptables-alerter-dvxpz"
Oct 30 18:33:27.329244 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329137    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-etc-selinux\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.329244 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329137    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-var-lib-kubelet\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.329244 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329163    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-run-ovn\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.329244 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329182    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-var-lib-kubelet\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.329244 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329188    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c816c10-1c32-4538-8724-834ebb405960-host-slash\") pod \"iptables-alerter-dvxpz\" (UID: \"0c816c10-1c32-4538-8724-834ebb405960\") " pod="openshift-network-operator/iptables-alerter-dvxpz"
Oct 30 18:33:27.329244 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329194    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-run-ovn\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.329244 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:27.329194    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret podName:6300215e-2302-47ed-bf47-05a165277c08 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:27.829170381 +0000 UTC m=+2.265434506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret") pod "global-pull-secret-syncer-gfs79" (UID: "6300215e-2302-47ed-bf47-05a165277c08") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:27.330084 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329228    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b1735618-e0f6-49b1-ade6-8fef300791a1-ovnkube-script-lib\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.330084 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329244    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zzhj\" (UniqueName: \"kubernetes.io/projected/b1735618-e0f6-49b1-ade6-8fef300791a1-kube-api-access-2zzhj\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.330084 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329258    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtpk7\" (UniqueName: \"kubernetes.io/projected/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-kube-api-access-vtpk7\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.330084 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329292    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-daemon-config\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.330084 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329300    2577 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 18:33:27.330084 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329335    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-cni-binary-copy\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.330084 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329358    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl92s\" (UniqueName: \"kubernetes.io/projected/0c816c10-1c32-4538-8724-834ebb405960-kube-api-access-zl92s\") pod \"iptables-alerter-dvxpz\" (UID: \"0c816c10-1c32-4538-8724-834ebb405960\") " pod="openshift-network-operator/iptables-alerter-dvxpz"
Oct 30 18:33:27.330084 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329385    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-system-cni-dir\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.330084 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329401    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-run-k8s-cni-cncf-io\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.330084 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329417    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-run-systemd\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.330084 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329466    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-run-systemd\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.330084 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329470    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-system-cni-dir\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.330084 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329467    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-run-k8s-cni-cncf-io\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.330084 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329498    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-sysconfig\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.330084 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329518    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-lib-modules\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.330084 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329532    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-run-netns\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.330084 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329547    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-kubernetes\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.330084 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329571    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-tuned\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.330989 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329579    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-sysconfig\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.330989 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329586    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-run-netns\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.330989 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329595    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.330989 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329617    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-hostroot\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.330989 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329628    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-kubernetes\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.330989 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329630    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-lib-modules\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.330989 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329632    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-var-lib-cni-bin\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.330989 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329663    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-var-lib-cni-bin\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.330989 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329675    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-run-netns\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.330989 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329693    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-hostroot\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.330989 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329702    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-log-socket\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.330989 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329720    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-run-netns\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.330989 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329727    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-device-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.330989 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329735    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b1735618-e0f6-49b1-ade6-8fef300791a1-ovnkube-script-lib\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.330989 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329750    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-cni-dir\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.330989 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329753    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-log-socket\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.330989 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329781    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-modprobe-d\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.330989 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329786    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-device-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.332088 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329800    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25k6c\" (UniqueName: \"kubernetes.io/projected/08a8e709-63af-4d71-b084-cf533ea620e5-kube-api-access-25k6c\") pod \"network-metrics-daemon-h5nm6\" (UID: \"08a8e709-63af-4d71-b084-cf533ea620e5\") " pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:27.332088 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329801    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-cni-dir\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.332088 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329818    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-socket-dir-parent\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.332088 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329842    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-kubelet\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.332088 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329861    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-modprobe-d\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.332088 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329867    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b1735618-e0f6-49b1-ade6-8fef300791a1-ovnkube-config\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.332088 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329870    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-socket-dir-parent\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.332088 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329890    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.332088 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329904    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-kubelet\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.332088 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329929    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.332088 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329968    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d3fa116f-e372-4a36-9a57-4832dc1440b3-hosts-file\") pod \"node-resolver-nr7xl\" (UID: \"d3fa116f-e372-4a36-9a57-4832dc1440b3\") " pod="openshift-dns/node-resolver-nr7xl"
Oct 30 18:33:27.332088 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.329998    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d3fa116f-e372-4a36-9a57-4832dc1440b3-hosts-file\") pod \"node-resolver-nr7xl\" (UID: \"d3fa116f-e372-4a36-9a57-4832dc1440b3\") " pod="openshift-dns/node-resolver-nr7xl"
Oct 30 18:33:27.332088 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330001    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.332088 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330035    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-run-ovn-kubernetes\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.332088 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330058    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b1735618-e0f6-49b1-ade6-8fef300791a1-env-overrides\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.332088 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330079    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.332088 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330081    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-run-ovn-kubernetes\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.332774 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330094    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px2j7\" (UniqueName: \"kubernetes.io/projected/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-kube-api-access-px2j7\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.332774 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330108    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-sysctl-d\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.332774 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330125    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-systemd\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.332774 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330146    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-host\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.332774 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330170    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b1735618-e0f6-49b1-ade6-8fef300791a1-ovn-node-metrics-cert\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.332774 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330195    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3443d97d-774b-4df3-abe3-3ca2d6b574b6-agent-certs\") pod \"konnectivity-agent-j4w9c\" (UID: \"3443d97d-774b-4df3-abe3-3ca2d6b574b6\") " pod="kube-system/konnectivity-agent-j4w9c"
Oct 30 18:33:27.332774 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330213    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-sysctl-d\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.332774 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330228    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs\") pod \"network-metrics-daemon-h5nm6\" (UID: \"08a8e709-63af-4d71-b084-cf533ea620e5\") " pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:27.332774 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330252    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6300215e-2302-47ed-bf47-05a165277c08-kubelet-config\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:27.332774 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330262    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-host\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.332774 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330274    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6300215e-2302-47ed-bf47-05a165277c08-dbus\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:27.332774 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330299    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdl78\" (UniqueName: \"kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78\") pod \"network-check-target-5gvjh\" (UID: \"a864943c-d7eb-4486-99a6-7961573b57d4\") " pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:27.332774 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330303    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-systemd\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.332774 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330325    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-system-cni-dir\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.332774 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330343    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.332774 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330355    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-cnibin\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.332774 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330378    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm96j\" (UniqueName: \"kubernetes.io/projected/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-kube-api-access-cm96j\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.333274 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330388    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b1735618-e0f6-49b1-ade6-8fef300791a1-ovnkube-config\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.333274 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330398    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6300215e-2302-47ed-bf47-05a165277c08-dbus\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:27.333274 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330401    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-sysctl-conf\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.333274 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330441    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7bfc481b-9824-43c5-acf0-436641114b03-host\") pod \"node-ca-8kldf\" (UID: \"7bfc481b-9824-43c5-acf0-436641114b03\") " pod="openshift-image-registry/node-ca-8kldf"
Oct 30 18:33:27.333274 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330444    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6300215e-2302-47ed-bf47-05a165277c08-kubelet-config\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:27.333274 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:27.330447    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:27.333274 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330469    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g5sn\" (UniqueName: \"kubernetes.io/projected/7bfc481b-9824-43c5-acf0-436641114b03-kube-api-access-5g5sn\") pod \"node-ca-8kldf\" (UID: \"7bfc481b-9824-43c5-acf0-436641114b03\") " pod="openshift-image-registry/node-ca-8kldf"
Oct 30 18:33:27.333274 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330573    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-node-log\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.333274 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:27.330585    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs podName:08a8e709-63af-4d71-b084-cf533ea620e5 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:27.830569128 +0000 UTC m=+2.266833238 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs") pod "network-metrics-daemon-h5nm6" (UID: "08a8e709-63af-4d71-b084-cf533ea620e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:27.333274 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330604    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-node-log\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.333274 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330614    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3443d97d-774b-4df3-abe3-3ca2d6b574b6-konnectivity-ca\") pod \"konnectivity-agent-j4w9c\" (UID: \"3443d97d-774b-4df3-abe3-3ca2d6b574b6\") " pod="kube-system/konnectivity-agent-j4w9c"
Oct 30 18:33:27.333274 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330615    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7bfc481b-9824-43c5-acf0-436641114b03-host\") pod \"node-ca-8kldf\" (UID: \"7bfc481b-9824-43c5-acf0-436641114b03\") " pod="openshift-image-registry/node-ca-8kldf"
Oct 30 18:33:27.333274 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330585    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b1735618-e0f6-49b1-ade6-8fef300791a1-env-overrides\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.333274 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330538    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-system-cni-dir\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.333274 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330686    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-sysctl-conf\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.333274 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330751    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-cnibin\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.333274 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330844    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-cnibin\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.333722 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330866    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms9kr\" (UniqueName: \"kubernetes.io/projected/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-kube-api-access-ms9kr\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.333722 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330897    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-cnibin\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.333722 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.330936    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-sys-fs\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.333722 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331003    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-sys-fs\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.333722 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331035    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-slash\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.333722 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331154    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-registration-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.333722 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331189    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7bfc481b-9824-43c5-acf0-436641114b03-serviceca\") pod \"node-ca-8kldf\" (UID: \"7bfc481b-9824-43c5-acf0-436641114b03\") " pod="openshift-image-registry/node-ca-8kldf"
Oct 30 18:33:27.333722 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331218    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3443d97d-774b-4df3-abe3-3ca2d6b574b6-konnectivity-ca\") pod \"konnectivity-agent-j4w9c\" (UID: \"3443d97d-774b-4df3-abe3-3ca2d6b574b6\") " pod="kube-system/konnectivity-agent-j4w9c"
Oct 30 18:33:27.333722 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331233    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-conf-dir\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.333722 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331254    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-registration-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.333722 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331259    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-systemd-units\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.333722 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331297    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-slash\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.333722 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331313    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-systemd-units\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.333722 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331345    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-conf-dir\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.333722 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331344    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-cni-netd\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.333722 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331413    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-cni-netd\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.333722 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331488    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99psk\" (UniqueName: \"kubernetes.io/projected/d3fa116f-e372-4a36-9a57-4832dc1440b3-kube-api-access-99psk\") pod \"node-resolver-nr7xl\" (UID: \"d3fa116f-e372-4a36-9a57-4832dc1440b3\") " pod="openshift-dns/node-resolver-nr7xl"
Oct 30 18:33:27.334212 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331514    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.334212 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331538    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-etc-kubernetes\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.334212 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331564    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-var-lib-openvswitch\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.334212 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331589    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-etc-openvswitch\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.334212 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331611    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-var-lib-kubelet\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.334212 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331636    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3fa116f-e372-4a36-9a57-4832dc1440b3-tmp-dir\") pod \"node-resolver-nr7xl\" (UID: \"d3fa116f-e372-4a36-9a57-4832dc1440b3\") " pod="openshift-dns/node-resolver-nr7xl"
Oct 30 18:33:27.334212 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331683    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-cni-binary-copy\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.334212 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331716    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0c816c10-1c32-4538-8724-834ebb405960-iptables-alerter-script\") pod \"iptables-alerter-dvxpz\" (UID: \"0c816c10-1c32-4538-8724-834ebb405960\") " pod="openshift-network-operator/iptables-alerter-dvxpz"
Oct 30 18:33:27.334212 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331712    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-etc-kubernetes\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.334212 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331802    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-var-lib-openvswitch\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.334212 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331846    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-etc-openvswitch\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.334212 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.331899    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-var-lib-kubelet\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.334212 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.332074    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.334212 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.332211    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3fa116f-e372-4a36-9a57-4832dc1440b3-tmp-dir\") pod \"node-resolver-nr7xl\" (UID: \"d3fa116f-e372-4a36-9a57-4832dc1440b3\") " pod="openshift-dns/node-resolver-nr7xl"
Oct 30 18:33:27.334212 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.332578    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-cni-binary-copy\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.334212 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.333004    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-tmp\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.334212 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.333054    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-tuned\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.334679 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.333155    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b1735618-e0f6-49b1-ade6-8fef300791a1-ovn-node-metrics-cert\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.334679 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.333254    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3443d97d-774b-4df3-abe3-3ca2d6b574b6-agent-certs\") pod \"konnectivity-agent-j4w9c\" (UID: \"3443d97d-774b-4df3-abe3-3ca2d6b574b6\") " pod="kube-system/konnectivity-agent-j4w9c"
Oct 30 18:33:27.339399 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.339370    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 18:33:27.339399 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.339376    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 18:33:27.339575 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.339561    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qb49q\""
Oct 30 18:33:27.339575 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.339573    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:33:27.339734 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.339619    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 18:33:27.339734 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.339664    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 18:33:27.339734 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.339671    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 18:33:27.339734 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.339677    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 18:33:27.339734 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.339664    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:33:27.339943 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.339762    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 18:33:27.339943 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.339786    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 18:33:27.339943 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.339803    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cgz7p\""
Oct 30 18:33:27.339943 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.339817    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 18:33:27.339943 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.339834    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cr7s6\""
Oct 30 18:33:27.339943 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.339879    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nh9hp\""
Oct 30 18:33:27.340555 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.340540    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.340866 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.340853    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.342494 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.342474    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7bfc481b-9824-43c5-acf0-436641114b03-serviceca\") pod \"node-ca-8kldf\" (UID: \"7bfc481b-9824-43c5-acf0-436641114b03\") " pod="openshift-image-registry/node-ca-8kldf"
Oct 30 18:33:27.342849 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.342831    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0c816c10-1c32-4538-8724-834ebb405960-iptables-alerter-script\") pod \"iptables-alerter-dvxpz\" (UID: \"0c816c10-1c32-4538-8724-834ebb405960\") " pod="openshift-network-operator/iptables-alerter-dvxpz"
Oct 30 18:33:27.391808 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.391787    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-px2j7\" (UniqueName: \"kubernetes.io/projected/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-kube-api-access-px2j7\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.394485 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:27.394449    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:33:27.394485 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:27.394467    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:33:27.394485 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:27.394477    2577 projected.go:196] Error preparing data for projected volume kube-api-access-zdl78 for pod openshift-network-diagnostics/network-check-target-5gvjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:27.394773 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:27.394533    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78 podName:a864943c-d7eb-4486-99a6-7961573b57d4 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:27.894520765 +0000 UTC m=+2.330784897 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zdl78" (UniqueName: "kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78") pod "network-check-target-5gvjh" (UID: "a864943c-d7eb-4486-99a6-7961573b57d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:27.396852 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.396828    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zzhj\" (UniqueName: \"kubernetes.io/projected/b1735618-e0f6-49b1-ade6-8fef300791a1-kube-api-access-2zzhj\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.396993 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.396976    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtpk7\" (UniqueName: \"kubernetes.io/projected/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-kube-api-access-vtpk7\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.397219 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.397200    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25k6c\" (UniqueName: \"kubernetes.io/projected/08a8e709-63af-4d71-b084-cf533ea620e5-kube-api-access-25k6c\") pod \"network-metrics-daemon-h5nm6\" (UID: \"08a8e709-63af-4d71-b084-cf533ea620e5\") " pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:27.397568 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.397554    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms9kr\" (UniqueName: \"kubernetes.io/projected/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-kube-api-access-ms9kr\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.401918 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.401898    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl92s\" (UniqueName: \"kubernetes.io/projected/0c816c10-1c32-4538-8724-834ebb405960-kube-api-access-zl92s\") pod \"iptables-alerter-dvxpz\" (UID: \"0c816c10-1c32-4538-8724-834ebb405960\") " pod="openshift-network-operator/iptables-alerter-dvxpz"
Oct 30 18:33:27.402352 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.402313    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm96j\" (UniqueName: \"kubernetes.io/projected/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-kube-api-access-cm96j\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.402446 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.402366    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99psk\" (UniqueName: \"kubernetes.io/projected/d3fa116f-e372-4a36-9a57-4832dc1440b3-kube-api-access-99psk\") pod \"node-resolver-nr7xl\" (UID: \"d3fa116f-e372-4a36-9a57-4832dc1440b3\") " pod="openshift-dns/node-resolver-nr7xl"
Oct 30 18:33:27.402446 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.402379    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g5sn\" (UniqueName: \"kubernetes.io/projected/7bfc481b-9824-43c5-acf0-436641114b03-kube-api-access-5g5sn\") pod \"node-ca-8kldf\" (UID: \"7bfc481b-9824-43c5-acf0-436641114b03\") " pod="openshift-image-registry/node-ca-8kldf"
Oct 30 18:33:27.521284 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.521219    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tzdf5"
Oct 30 18:33:27.547930 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.547903    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j4w9c"
Oct 30 18:33:27.553357 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.553338    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:27.564533 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:27.564502    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd085b0a4a8eed9a1bd72245b7d7a6e5e.slice/crio-20eb7acf3821a2acf3f28ee6fde8dd72950a5c8a215d3e8d6c9bf99861bb6720 WatchSource:0}: Error finding container 20eb7acf3821a2acf3f28ee6fde8dd72950a5c8a215d3e8d6c9bf99861bb6720: Status 404 returned error can't find the container with id 20eb7acf3821a2acf3f28ee6fde8dd72950a5c8a215d3e8d6c9bf99861bb6720
Oct 30 18:33:27.564930 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:27.564903    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff8defe0c0f1527a0efd6540e2b0bdc3.slice/crio-212e735c50505ff6ea0b59b17796bc0e130cfb49f55ca39ae1055e8ada6c7223 WatchSource:0}: Error finding container 212e735c50505ff6ea0b59b17796bc0e130cfb49f55ca39ae1055e8ada6c7223: Status 404 returned error can't find the container with id 212e735c50505ff6ea0b59b17796bc0e130cfb49f55ca39ae1055e8ada6c7223
Oct 30 18:33:27.566078 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:27.566057    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1735618_e0f6_49b1_ade6_8fef300791a1.slice/crio-81477958f53c63164f8c06fe47dbf411f367635a5caedb46f7fb6d02c7b42abb WatchSource:0}: Error finding container 81477958f53c63164f8c06fe47dbf411f367635a5caedb46f7fb6d02c7b42abb: Status 404 returned error can't find the container with id 81477958f53c63164f8c06fe47dbf411f367635a5caedb46f7fb6d02c7b42abb
Oct 30 18:33:27.566554 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:27.566539    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3443d97d_774b_4df3_abe3_3ca2d6b574b6.slice/crio-00505abb739fe1fb635c5bef9337ca21aceb3738ce32d4ed32a584a25f1a2bb2 WatchSource:0}: Error finding container 00505abb739fe1fb635c5bef9337ca21aceb3738ce32d4ed32a584a25f1a2bb2: Status 404 returned error can't find the container with id 00505abb739fe1fb635c5bef9337ca21aceb3738ce32d4ed32a584a25f1a2bb2
Oct 30 18:33:27.570042 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.570026    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:33:27.573332 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.573317    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:33:27.573433 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.573417    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:33:27.582773 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:27.582755    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3f102b3_d7bb_4ae4_b3cf_c20dd774804d.slice/crio-bed9f04f02d688cb07374d7c4b0e17a6ccfe8e6514e496d3b9d790e017bff4f6 WatchSource:0}: Error finding container bed9f04f02d688cb07374d7c4b0e17a6ccfe8e6514e496d3b9d790e017bff4f6: Status 404 returned error can't find the container with id bed9f04f02d688cb07374d7c4b0e17a6ccfe8e6514e496d3b9d790e017bff4f6
Oct 30 18:33:27.584836 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:27.584819    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb0fef6c_8c89_40af_bd03_99cb7035a3f2.slice/crio-f5a1c67be6c1aecd39285c7c43774217e0fb6a8d6640de48c8262494d7f8c4b2 WatchSource:0}: Error finding container f5a1c67be6c1aecd39285c7c43774217e0fb6a8d6640de48c8262494d7f8c4b2: Status 404 returned error can't find the container with id f5a1c67be6c1aecd39285c7c43774217e0fb6a8d6640de48c8262494d7f8c4b2
Oct 30 18:33:27.585501 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.585486    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nr7xl"
Oct 30 18:33:27.593828 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:27.593810    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3fa116f_e372_4a36_9a57_4832dc1440b3.slice/crio-6f003228b1d30cf5260857ba5f58754dc25cc6b038c6d90fcfab7263e4b95825 WatchSource:0}: Error finding container 6f003228b1d30cf5260857ba5f58754dc25cc6b038c6d90fcfab7263e4b95825: Status 404 returned error can't find the container with id 6f003228b1d30cf5260857ba5f58754dc25cc6b038c6d90fcfab7263e4b95825
Oct 30 18:33:27.616815 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.616800    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:33:27.617863 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.617849    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dvxpz"
Oct 30 18:33:27.621888 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.621875    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8kldf"
Oct 30 18:33:27.625823 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:27.625804    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1cec39c_0df9_4b68_bf4c_8c58c65ebc7a.slice/crio-6855c78ac71ded17d8193720a1e1dd2bddeab009c2df60b0f48db65b03d540c2 WatchSource:0}: Error finding container 6855c78ac71ded17d8193720a1e1dd2bddeab009c2df60b0f48db65b03d540c2: Status 404 returned error can't find the container with id 6855c78ac71ded17d8193720a1e1dd2bddeab009c2df60b0f48db65b03d540c2
Oct 30 18:33:27.628866 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:27.628784    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c816c10_1c32_4538_8724_834ebb405960.slice/crio-3a44400ad69e02975e061a9f803ace91f50a86644a2df50b0c7040ec8f11aebd WatchSource:0}: Error finding container 3a44400ad69e02975e061a9f803ace91f50a86644a2df50b0c7040ec8f11aebd: Status 404 returned error can't find the container with id 3a44400ad69e02975e061a9f803ace91f50a86644a2df50b0c7040ec8f11aebd
Oct 30 18:33:27.631880 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:27.631862    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bfc481b_9824_43c5_acf0_436641114b03.slice/crio-8eba0dd7e02915db6673073d0511c7ef49110c853ecfc4e90032da3eeb645118 WatchSource:0}: Error finding container 8eba0dd7e02915db6673073d0511c7ef49110c853ecfc4e90032da3eeb645118: Status 404 returned error can't find the container with id 8eba0dd7e02915db6673073d0511c7ef49110c853ecfc4e90032da3eeb645118
Oct 30 18:33:27.834689 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.834607    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs\") pod \"network-metrics-daemon-h5nm6\" (UID: \"08a8e709-63af-4d71-b084-cf533ea620e5\") " pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:27.834689 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.834670    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:27.834914 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:27.834764    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:27.834914 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:27.834785    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:27.834914 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:27.834830    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs podName:08a8e709-63af-4d71-b084-cf533ea620e5 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:28.834811679 +0000 UTC m=+3.271075804 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs") pod "network-metrics-daemon-h5nm6" (UID: "08a8e709-63af-4d71-b084-cf533ea620e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:27.834914 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:27.834846    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret podName:6300215e-2302-47ed-bf47-05a165277c08 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:28.834838766 +0000 UTC m=+3.271102876 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret") pod "global-pull-secret-syncer-gfs79" (UID: "6300215e-2302-47ed-bf47-05a165277c08") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:27.935667 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:27.935629    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdl78\" (UniqueName: \"kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78\") pod \"network-check-target-5gvjh\" (UID: \"a864943c-d7eb-4486-99a6-7961573b57d4\") " pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:27.936335 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:27.935779    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:33:27.936335 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:27.935798    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:33:27.936335 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:27.935807    2577 projected.go:196] Error preparing data for projected volume kube-api-access-zdl78 for pod openshift-network-diagnostics/network-check-target-5gvjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:27.936335 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:27.935856    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78 podName:a864943c-d7eb-4486-99a6-7961573b57d4 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:28.935841646 +0000 UTC m=+3.372105755 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zdl78" (UniqueName: "kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78") pod "network-check-target-5gvjh" (UID: "a864943c-d7eb-4486-99a6-7961573b57d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:28.049927 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:28.049877    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dvxpz" event={"ID":"0c816c10-1c32-4538-8724-834ebb405960","Type":"ContainerStarted","Data":"3a44400ad69e02975e061a9f803ace91f50a86644a2df50b0c7040ec8f11aebd"}
Oct 30 18:33:28.050958 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:28.050915    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nr7xl" event={"ID":"d3fa116f-e372-4a36-9a57-4832dc1440b3","Type":"ContainerStarted","Data":"6f003228b1d30cf5260857ba5f58754dc25cc6b038c6d90fcfab7263e4b95825"}
Oct 30 18:33:28.052061 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:28.052008    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5" event={"ID":"fb0fef6c-8c89-40af-bd03-99cb7035a3f2","Type":"ContainerStarted","Data":"f5a1c67be6c1aecd39285c7c43774217e0fb6a8d6640de48c8262494d7f8c4b2"}
Oct 30 18:33:28.053089 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:28.053050    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j4w9c" event={"ID":"3443d97d-774b-4df3-abe3-3ca2d6b574b6","Type":"ContainerStarted","Data":"00505abb739fe1fb635c5bef9337ca21aceb3738ce32d4ed32a584a25f1a2bb2"}
Oct 30 18:33:28.054182 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:28.054154    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8kldf" event={"ID":"7bfc481b-9824-43c5-acf0-436641114b03","Type":"ContainerStarted","Data":"8eba0dd7e02915db6673073d0511c7ef49110c853ecfc4e90032da3eeb645118"}
Oct 30 18:33:28.055263 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:28.055238    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jpvz" event={"ID":"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a","Type":"ContainerStarted","Data":"6855c78ac71ded17d8193720a1e1dd2bddeab009c2df60b0f48db65b03d540c2"}
Oct 30 18:33:28.056320 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:28.056299    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nnld2" event={"ID":"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d","Type":"ContainerStarted","Data":"bed9f04f02d688cb07374d7c4b0e17a6ccfe8e6514e496d3b9d790e017bff4f6"}
Oct 30 18:33:28.057569 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:28.057537    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg" event={"ID":"b1735618-e0f6-49b1-ade6-8fef300791a1","Type":"ContainerStarted","Data":"81477958f53c63164f8c06fe47dbf411f367635a5caedb46f7fb6d02c7b42abb"}
Oct 30 18:33:28.058643 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:28.058624    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal" event={"ID":"d085b0a4a8eed9a1bd72245b7d7a6e5e","Type":"ContainerStarted","Data":"20eb7acf3821a2acf3f28ee6fde8dd72950a5c8a215d3e8d6c9bf99861bb6720"}
Oct 30 18:33:28.059687 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:28.059665    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal" event={"ID":"ff8defe0c0f1527a0efd6540e2b0bdc3","Type":"ContainerStarted","Data":"212e735c50505ff6ea0b59b17796bc0e130cfb49f55ca39ae1055e8ada6c7223"}
Oct 30 18:33:28.131819 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:28.131757    2577 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:33:28.151895 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:28.151874    2577 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:33:28.240650 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:28.240596    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:28:27 +0000 UTC" deadline="2028-04-20 06:50:02.938087592 +0000 UTC"
Oct 30 18:33:28.240650 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:28.240643    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12900h16m34.697449661s"
Oct 30 18:33:28.428328 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:33:28.428301    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5c45faf_cafe_4ee2_9faa_f8686b13c4bd.slice/crio-f942e7ffa5a1f0246723ff144901e3c94c2e3616b065473354c0ef5804b08343 WatchSource:0}: Error finding container f942e7ffa5a1f0246723ff144901e3c94c2e3616b065473354c0ef5804b08343: Status 404 returned error can't find the container with id f942e7ffa5a1f0246723ff144901e3c94c2e3616b065473354c0ef5804b08343
Oct 30 18:33:28.465782 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:28.465757    2577 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:33:28.845384 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:28.845298    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:28.845384 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:28.845378    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs\") pod \"network-metrics-daemon-h5nm6\" (UID: \"08a8e709-63af-4d71-b084-cf533ea620e5\") " pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:28.845620 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:28.845500    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:28.845620 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:28.845564    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs podName:08a8e709-63af-4d71-b084-cf533ea620e5 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:30.845543283 +0000 UTC m=+5.281807397 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs") pod "network-metrics-daemon-h5nm6" (UID: "08a8e709-63af-4d71-b084-cf533ea620e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:28.845745 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:28.845630    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:28.845745 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:28.845664    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret podName:6300215e-2302-47ed-bf47-05a165277c08 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:30.845652586 +0000 UTC m=+5.281916699 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret") pod "global-pull-secret-syncer-gfs79" (UID: "6300215e-2302-47ed-bf47-05a165277c08") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:28.946574 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:28.946538    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdl78\" (UniqueName: \"kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78\") pod \"network-check-target-5gvjh\" (UID: \"a864943c-d7eb-4486-99a6-7961573b57d4\") " pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:28.947189 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:28.946752    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:33:28.947189 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:28.946772    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:33:28.947189 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:28.946785    2577 projected.go:196] Error preparing data for projected volume kube-api-access-zdl78 for pod openshift-network-diagnostics/network-check-target-5gvjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:28.947189 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:28.946843    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78 podName:a864943c-d7eb-4486-99a6-7961573b57d4 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:30.946825046 +0000 UTC m=+5.383089163 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zdl78" (UniqueName: "kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78") pod "network-check-target-5gvjh" (UID: "a864943c-d7eb-4486-99a6-7961573b57d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:29.045648 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:29.045613    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:29.045832 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:29.045708    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:29.045902 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:29.045878    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:33:29.046049 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:29.045999    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:33:29.046137 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:29.046062    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:29.046198 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:29.046139    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:33:29.114670 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:29.114587    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tzdf5" event={"ID":"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd","Type":"ContainerStarted","Data":"f942e7ffa5a1f0246723ff144901e3c94c2e3616b065473354c0ef5804b08343"}
Oct 30 18:33:29.242482 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:29.241525    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:28:27 +0000 UTC" deadline="2028-06-30 08:32:49.947906447 +0000 UTC"
Oct 30 18:33:29.242482 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:29.241565    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14605h59m20.706346015s"
Oct 30 18:33:30.866937 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:30.866877    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs\") pod \"network-metrics-daemon-h5nm6\" (UID: \"08a8e709-63af-4d71-b084-cf533ea620e5\") " pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:30.867653 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:30.866986    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:30.867653 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:30.867148    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:30.867653 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:30.867213    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret podName:6300215e-2302-47ed-bf47-05a165277c08 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:34.867194855 +0000 UTC m=+9.303458967 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret") pod "global-pull-secret-syncer-gfs79" (UID: "6300215e-2302-47ed-bf47-05a165277c08") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:30.867653 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:30.867287    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:30.867653 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:30.867323    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs podName:08a8e709-63af-4d71-b084-cf533ea620e5 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:34.867310194 +0000 UTC m=+9.303574314 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs") pod "network-metrics-daemon-h5nm6" (UID: "08a8e709-63af-4d71-b084-cf533ea620e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:30.968131 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:30.968086    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdl78\" (UniqueName: \"kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78\") pod \"network-check-target-5gvjh\" (UID: \"a864943c-d7eb-4486-99a6-7961573b57d4\") " pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:30.968330 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:30.968306    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:33:30.968330 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:30.968326    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:33:30.968437 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:30.968339    2577 projected.go:196] Error preparing data for projected volume kube-api-access-zdl78 for pod openshift-network-diagnostics/network-check-target-5gvjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:30.968437 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:30.968398    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78 podName:a864943c-d7eb-4486-99a6-7961573b57d4 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:34.968379808 +0000 UTC m=+9.404643921 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zdl78" (UniqueName: "kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78") pod "network-check-target-5gvjh" (UID: "a864943c-d7eb-4486-99a6-7961573b57d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:31.046990 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:31.046439    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:31.046990 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:31.046602    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:33:31.046990 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:31.046686    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:31.046990 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:31.046765    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:33:31.046990 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:31.046825    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:31.046990 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:31.046892    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:33:32.146552 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:32.146298    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jpvz" event={"ID":"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a","Type":"ContainerStarted","Data":"10f25d7d2a0d5a40f6107a0c66f7e75a751838b29911e73eeb02fc255766ebe6"}
Oct 30 18:33:33.046378 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:33.046254    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:33.046554 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:33.046402    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:33:33.046554 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:33.046475    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:33.046554 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:33.046548    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:33:33.046740 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:33.046586    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:33.046814 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:33.046787    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:33:33.156656 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:33.155893    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j4w9c" event={"ID":"3443d97d-774b-4df3-abe3-3ca2d6b574b6","Type":"ContainerStarted","Data":"0e59162038ccfb91ed680dde94463a453fa5097fff45671fbb5786cfe6ecc225"}
Oct 30 18:33:33.163430 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:33.162282    2577 generic.go:356] "Generic (PLEG): container finished" podID="a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a" containerID="10f25d7d2a0d5a40f6107a0c66f7e75a751838b29911e73eeb02fc255766ebe6" exitCode=0
Oct 30 18:33:33.163430 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:33.162451    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jpvz" event={"ID":"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a","Type":"ContainerDied","Data":"10f25d7d2a0d5a40f6107a0c66f7e75a751838b29911e73eeb02fc255766ebe6"}
Oct 30 18:33:33.176807 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:33.176251    2577 generic.go:356] "Generic (PLEG): container finished" podID="d085b0a4a8eed9a1bd72245b7d7a6e5e" containerID="9c94177951b44bc3c0bdddcede8570f6c7214960dc3045f8b404949e9dc01728" exitCode=0
Oct 30 18:33:33.176807 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:33.176293    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal" event={"ID":"d085b0a4a8eed9a1bd72245b7d7a6e5e","Type":"ContainerDied","Data":"9c94177951b44bc3c0bdddcede8570f6c7214960dc3045f8b404949e9dc01728"}
Oct 30 18:33:33.284661 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:33.284475    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-j4w9c" podStartSLOduration=2.200106875 podStartE2EDuration="7.284456665s" totalImagesPullingTime="5.08434979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:33.284160964 +0000 UTC m=+7.720425098" watchObservedRunningTime="2026-10-30 18:33:33.284456665 +0000 UTC m=+7.720720825"
Oct 30 18:33:34.186226 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:34.186147    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8kldf" event={"ID":"7bfc481b-9824-43c5-acf0-436641114b03","Type":"ContainerStarted","Data":"ee4ef852aaca49ee0b9e71491414b9eabac126383250d3df568781740e09ae81"}
Oct 30 18:33:34.194462 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:34.194403    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal" event={"ID":"d085b0a4a8eed9a1bd72245b7d7a6e5e","Type":"ContainerStarted","Data":"bc3cb9b6374a791c53326adc2b6972d9424f180e393edb538789b4338c326d51"}
Oct 30 18:33:34.200180 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:34.200091    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal" event={"ID":"ff8defe0c0f1527a0efd6540e2b0bdc3","Type":"ContainerStarted","Data":"d5bef21a62e16b6a29db646890371946d3b4a1582c3ea77a0de9aff45391802f"}
Oct 30 18:33:34.312989 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:34.312927    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8kldf" podStartSLOduration=2.105105814 podStartE2EDuration="8.312906701s" totalImagesPullingTime="6.207800887s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:34.312247255 +0000 UTC m=+8.748511390" watchObservedRunningTime="2026-10-30 18:33:34.312906701 +0000 UTC m=+8.749170838"
Oct 30 18:33:34.568696 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:34.568630    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal" podStartSLOduration=8.568608809 podStartE2EDuration="8.568608809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:33:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:34.441129497 +0000 UTC m=+8.877393632" watchObservedRunningTime="2026-10-30 18:33:34.568608809 +0000 UTC m=+9.004872944"
Oct 30 18:33:34.910514 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:34.910478    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs\") pod \"network-metrics-daemon-h5nm6\" (UID: \"08a8e709-63af-4d71-b084-cf533ea620e5\") " pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:34.910683 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:34.910559    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:34.910752 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:34.910711    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:34.910785 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:34.910772    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret podName:6300215e-2302-47ed-bf47-05a165277c08 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:42.910754526 +0000 UTC m=+17.347018658 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret") pod "global-pull-secret-syncer-gfs79" (UID: "6300215e-2302-47ed-bf47-05a165277c08") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:34.910850 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:34.910824    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:34.910899 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:34.910859    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs podName:08a8e709-63af-4d71-b084-cf533ea620e5 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:42.910848462 +0000 UTC m=+17.347112586 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs") pod "network-metrics-daemon-h5nm6" (UID: "08a8e709-63af-4d71-b084-cf533ea620e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:35.011615 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:35.011253    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdl78\" (UniqueName: \"kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78\") pod \"network-check-target-5gvjh\" (UID: \"a864943c-d7eb-4486-99a6-7961573b57d4\") " pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:35.011615 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:35.011448    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:33:35.011615 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:35.011468    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:33:35.011615 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:35.011481    2577 projected.go:196] Error preparing data for projected volume kube-api-access-zdl78 for pod openshift-network-diagnostics/network-check-target-5gvjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:35.011615 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:35.011536    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78 podName:a864943c-d7eb-4486-99a6-7961573b57d4 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:43.011518685 +0000 UTC m=+17.447782810 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zdl78" (UniqueName: "kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78") pod "network-check-target-5gvjh" (UID: "a864943c-d7eb-4486-99a6-7961573b57d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:35.046265 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:35.045804    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:35.046265 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:35.045928    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:33:35.046265 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:35.046016    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:35.046265 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:35.046090    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:33:35.046265 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:35.046154    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:35.046265 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:35.046232    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:33:35.205708 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:35.205665    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5" event={"ID":"fb0fef6c-8c89-40af-bd03-99cb7035a3f2","Type":"ContainerStarted","Data":"b28742091d065a002f21e82a1fe734a76620663248c08dcec183e3897f0a21b7"}
Oct 30 18:33:36.256480 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:36.256446    2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-j4w9c"
Oct 30 18:33:36.257503 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:36.257328    2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-j4w9c"
Oct 30 18:33:36.375556 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:36.375147    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal" podStartSLOduration=10.375125145 podStartE2EDuration="10.375125145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:33:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:34.568968633 +0000 UTC m=+9.005232769" watchObservedRunningTime="2026-10-30 18:33:36.375125145 +0000 UTC m=+10.811389276"
Oct 30 18:33:36.744407 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:36.744325    2577 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 18:33:36.992153 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:36.991812    2577 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:33:36.744355073Z","UUID":"6db31a9b-e641-46e8-ab9f-a88165821f66","Handler":null,"Name":"","Endpoint":""}
Oct 30 18:33:36.994899 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:36.994874    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 18:33:36.994899 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:36.994899    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 18:33:37.046022 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:37.045993    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:37.046170 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:37.046027    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:37.046170 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:37.045993    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:37.046170 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:37.046115    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:33:37.046359 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:37.046333    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:33:37.047382 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:37.047344    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:33:37.212079 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:37.212002    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nr7xl" event={"ID":"d3fa116f-e372-4a36-9a57-4832dc1440b3","Type":"ContainerStarted","Data":"3d14468d466c8240b1fd91af9640c1ef08b13061953c44aa8479e0523ecb5b75"}
Oct 30 18:33:37.214420 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:37.214386    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5" event={"ID":"fb0fef6c-8c89-40af-bd03-99cb7035a3f2","Type":"ContainerStarted","Data":"96a5b2641c882200aef42122d3dd31b4d4e7fda7829aa3b93b4d69b45bc5cc6f"}
Oct 30 18:33:37.216336 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:37.216302    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tzdf5" event={"ID":"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd","Type":"ContainerStarted","Data":"24aa1ccdd02c18d8f0c7ce21bd554b7649eae61af6c86cf6a9fe5635a4e13e5b"}
Oct 30 18:33:37.335617 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:37.335559    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-nr7xl" podStartSLOduration=2.066486685 podStartE2EDuration="11.335541065s" totalImagesPullingTime="9.26905438s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:37.335260375 +0000 UTC m=+11.771524505" watchObservedRunningTime="2026-10-30 18:33:37.335541065 +0000 UTC m=+11.771805233"
Oct 30 18:33:37.461382 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:37.461324    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-tzdf5" podStartSLOduration=3.038764799 podStartE2EDuration="11.461307838s" totalImagesPullingTime="8.422543039s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:37.460632909 +0000 UTC m=+11.896897051" watchObservedRunningTime="2026-10-30 18:33:37.461307838 +0000 UTC m=+11.897571968"
Oct 30 18:33:38.221639 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:38.221378    2577 generic.go:356] "Generic (PLEG): container finished" podID="a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a" containerID="00bb5b7ca5c534e5f09448453fcfdd026c68fb2596b29e88b44cc0f90e10c164" exitCode=0
Oct 30 18:33:38.221853 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:38.221431    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jpvz" event={"ID":"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a","Type":"ContainerDied","Data":"00bb5b7ca5c534e5f09448453fcfdd026c68fb2596b29e88b44cc0f90e10c164"}
Oct 30 18:33:39.046286 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:39.046259    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:39.046664 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:39.046288    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:39.046664 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:39.046293    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:39.046664 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:39.046360    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:33:39.046664 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:39.046418    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:33:39.046664 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:39.046492    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:33:39.225844 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:39.225815    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nnld2" event={"ID":"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d","Type":"ContainerStarted","Data":"aa5acdd74df8683512db66c0af1a4fb2dc41db304b13acae7b81c05a0926b335"}
Oct 30 18:33:39.228007 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:39.227666    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dvxpz" event={"ID":"0c816c10-1c32-4538-8724-834ebb405960","Type":"ContainerStarted","Data":"96d939994eb32607d5127d60f37b462c88f84daf383fd7a89dd18e4e2f1d6704"}
Oct 30 18:33:39.230196 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:39.230169    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5" event={"ID":"fb0fef6c-8c89-40af-bd03-99cb7035a3f2","Type":"ContainerStarted","Data":"2e8291f755d454455081e9b800640cb2048b99220a69bd1d275feb06c3747fb4"}
Oct 30 18:33:39.345547 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:39.345444    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nnld2" podStartSLOduration=1.850600757 podStartE2EDuration="13.345425395s" totalImagesPullingTime="11.494824638s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:39.345182682 +0000 UTC m=+13.781446813" watchObservedRunningTime="2026-10-30 18:33:39.345425395 +0000 UTC m=+13.781689527"
Oct 30 18:33:39.471916 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:39.471864    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5" podStartSLOduration=3.313878653 podStartE2EDuration="13.471851819s" totalImagesPullingTime="10.157973166s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:33:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:39.471201475 +0000 UTC m=+13.907465606" watchObservedRunningTime="2026-10-30 18:33:39.471851819 +0000 UTC m=+13.908115987"
Oct 30 18:33:40.235136 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:40.235098    2577 generic.go:356] "Generic (PLEG): container finished" podID="a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a" containerID="1f450a1906406d89aa98aee291c7232d4ccbd52b478be1e75b599bac05f23d80" exitCode=0
Oct 30 18:33:40.235795 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:40.235204    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jpvz" event={"ID":"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a","Type":"ContainerDied","Data":"1f450a1906406d89aa98aee291c7232d4ccbd52b478be1e75b599bac05f23d80"}
Oct 30 18:33:40.461290 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:40.461230    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dvxpz" podStartSLOduration=5.29517408 podStartE2EDuration="14.461210256s" totalImagesPullingTime="9.166036176s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:39.588292818 +0000 UTC m=+14.024556950" watchObservedRunningTime="2026-10-30 18:33:40.461210256 +0000 UTC m=+14.897474387"
Oct 30 18:33:41.045904 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:41.045864    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:41.045904 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:41.045890    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:41.046154 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:41.046033    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:33:41.046154 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:41.046113    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:33:41.046154 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:41.046135    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:41.046293 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:41.046240    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:33:42.242713 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:42.242625    2577 generic.go:356] "Generic (PLEG): container finished" podID="a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a" containerID="85c487d7d0b52c1ea626aa5e7ca616c6be6973bbd5bedaf15dab859d20a8487a" exitCode=0
Oct 30 18:33:42.242713 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:42.242686    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jpvz" event={"ID":"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a","Type":"ContainerDied","Data":"85c487d7d0b52c1ea626aa5e7ca616c6be6973bbd5bedaf15dab859d20a8487a"}
Oct 30 18:33:42.969388 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:42.969350    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs\") pod \"network-metrics-daemon-h5nm6\" (UID: \"08a8e709-63af-4d71-b084-cf533ea620e5\") " pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:42.969558 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:42.969412    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:42.969558 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:42.969515    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:42.969558 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:42.969515    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:42.969700 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:42.969581    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs podName:08a8e709-63af-4d71-b084-cf533ea620e5 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:58.969563919 +0000 UTC m=+33.405828031 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs") pod "network-metrics-daemon-h5nm6" (UID: "08a8e709-63af-4d71-b084-cf533ea620e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:42.969700 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:42.969599    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret podName:6300215e-2302-47ed-bf47-05a165277c08 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:58.969590338 +0000 UTC m=+33.405854449 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret") pod "global-pull-secret-syncer-gfs79" (UID: "6300215e-2302-47ed-bf47-05a165277c08") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:43.045572 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:43.045545    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:43.045714 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:43.045633    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:43.045714 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:43.045652    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:43.045794 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:43.045743    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:33:43.045833 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:43.045819    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:33:43.045871 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:43.045861    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:33:43.069788 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:43.069749    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdl78\" (UniqueName: \"kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78\") pod \"network-check-target-5gvjh\" (UID: \"a864943c-d7eb-4486-99a6-7961573b57d4\") " pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:43.069930 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:43.069908    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:33:43.069930 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:43.069926    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:33:43.070040 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:43.069938    2577 projected.go:196] Error preparing data for projected volume kube-api-access-zdl78 for pod openshift-network-diagnostics/network-check-target-5gvjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:43.070040 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:43.070021    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78 podName:a864943c-d7eb-4486-99a6-7961573b57d4 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:59.070003388 +0000 UTC m=+33.506267501 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zdl78" (UniqueName: "kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78") pod "network-check-target-5gvjh" (UID: "a864943c-d7eb-4486-99a6-7961573b57d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:45.046149 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:45.046127    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:45.046507 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:45.046234    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:33:45.046507 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:45.046282    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:45.046507 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:45.046325    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:33:45.046507 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:45.046356    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:45.046507 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:45.046395    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:33:45.252441 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:45.252417    2577 generic.go:356] "Generic (PLEG): container finished" podID="a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a" containerID="ed3d244e3f6496b096d0728644592305e87c7e17d7a3f234286596dbbcb6ddc6" exitCode=0
Oct 30 18:33:45.252537 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:45.252501    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jpvz" event={"ID":"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a","Type":"ContainerDied","Data":"ed3d244e3f6496b096d0728644592305e87c7e17d7a3f234286596dbbcb6ddc6"}
Oct 30 18:33:45.255354 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:45.255329    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg" event={"ID":"b1735618-e0f6-49b1-ade6-8fef300791a1","Type":"ContainerStarted","Data":"00ac1d847b6b6a2614486a4df7e5d84f1a55bc80823e5b8a4c3aaa301fecfb28"}
Oct 30 18:33:45.255451 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:45.255392    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg" event={"ID":"b1735618-e0f6-49b1-ade6-8fef300791a1","Type":"ContainerStarted","Data":"0071c6b3c9c2f370b76fbe336569e53a264024b7497e1be77b30c4933d14fe48"}
Oct 30 18:33:45.255451 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:45.255440    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg" event={"ID":"b1735618-e0f6-49b1-ade6-8fef300791a1","Type":"ContainerStarted","Data":"00d2941117832c1c2ab7b6693e9b4f2974f7ba9d570ef1b747c322b2ce844b24"}
Oct 30 18:33:45.255512 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:45.255493    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg" event={"ID":"b1735618-e0f6-49b1-ade6-8fef300791a1","Type":"ContainerStarted","Data":"5c899946312d2660550f7f89962a9b80e8276c47002491f27ec72eab7b87e10a"}
Oct 30 18:33:45.255555 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:45.255540    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg" event={"ID":"b1735618-e0f6-49b1-ade6-8fef300791a1","Type":"ContainerStarted","Data":"c0c9eb1d818a67c7928cdc3ba4934e0075ceb6ed88bffc1d50cf92e194e14c71"}
Oct 30 18:33:46.260583 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:46.260396    2577 generic.go:356] "Generic (PLEG): container finished" podID="a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a" containerID="520723f61c5004eac96e043770d58d711e55738cbc755afae571f2242dba868e" exitCode=0
Oct 30 18:33:46.261058 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:46.260475    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jpvz" event={"ID":"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a","Type":"ContainerDied","Data":"520723f61c5004eac96e043770d58d711e55738cbc755afae571f2242dba868e"}
Oct 30 18:33:46.266266 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:46.266244    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg" event={"ID":"b1735618-e0f6-49b1-ade6-8fef300791a1","Type":"ContainerStarted","Data":"2c7660416d67c8fcb594d3fbc1227ab1791fcc25b3c7f33127703b91f6af0c87"}
Oct 30 18:33:47.046044 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:47.046009    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:47.046044 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:47.046037    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:47.046267 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:47.046004    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:47.046267 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:47.046151    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:33:47.046267 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:47.046204    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:33:47.046382 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:47.046277    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:33:47.272107 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:47.272067    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jpvz" event={"ID":"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a","Type":"ContainerStarted","Data":"3551766944a51160f334941f9e78cf00a2902225804ca89ad9ac93f6ab67d253"}
Oct 30 18:33:47.397044 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:47.396984    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8jpvz" podStartSLOduration=7.890556304 podStartE2EDuration="21.396944663s" totalImagesPullingTime="13.506388359s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:33:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:47.396687264 +0000 UTC m=+21.832951420" watchObservedRunningTime="2026-10-30 18:33:47.396944663 +0000 UTC m=+21.833208796"
Oct 30 18:33:48.277076 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:48.277039    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg" event={"ID":"b1735618-e0f6-49b1-ade6-8fef300791a1","Type":"ContainerStarted","Data":"b210540e2c6d45fdfcf8468c504f98e2a4a8856e0f98684aa49a97071ddbddd6"}
Oct 30 18:33:48.338536 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:48.338513    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:33:48.569070 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:48.568990    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:33:48.814819 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:48.814790    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:33:49.045716 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:49.045645    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:49.045888 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:49.045733    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:49.045888 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:49.045821    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:49.045888 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:49.045831    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:33:49.046059 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:49.045964    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:33:49.046059 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:49.046046    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:33:49.451394 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:49.451367    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:33:49.654525 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:49.654491    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:33:50.014261 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:50.014234    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-j4w9c"
Oct 30 18:33:50.014416 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:50.014353    2577 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 18:33:50.014817 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:50.014797    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-j4w9c"
Oct 30 18:33:50.253204 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:50.253159    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:33:50.285119 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:50.285053    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg" event={"ID":"b1735618-e0f6-49b1-ade6-8fef300791a1","Type":"ContainerStarted","Data":"3fa1dd3c7f1e99c24fcf717609aeb1c9449884404ed97155d994a93e5ceebaae"}
Oct 30 18:33:50.495538 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:50.495489    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg" podStartSLOduration=7.135485373 podStartE2EDuration="24.495477597s" totalImagesPullingTime="17.359992224s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:50.467939707 +0000 UTC m=+24.904203837" watchObservedRunningTime="2026-10-30 18:33:50.495477597 +0000 UTC m=+24.931741726"
Oct 30 18:33:51.045576 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:51.045546    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:51.045576 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:51.045570    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:51.045762 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:51.045548    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:51.045762 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:51.045653    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:33:51.045762 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:51.045727    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:33:51.045852 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:51.045793    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:33:51.287707 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:51.287679    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:51.287883 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:51.287756    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:51.287883 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:51.287792    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:51.302701 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:51.302639    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:51.302851 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:51.302838    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:33:52.032128 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:52.032096    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:33:52.196740 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:52.196650    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5gvjh"]
Oct 30 18:33:52.196895 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:52.196746    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:52.196895 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:52.196833    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:33:52.200115 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:52.200094    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gfs79"]
Oct 30 18:33:52.200230 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:52.200168    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:52.200279 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:52.200248    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:33:52.200869 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:52.200849    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-h5nm6"]
Oct 30 18:33:52.200939 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:52.200928    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:52.201031 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:52.201015    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:33:52.259241 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:52.259213    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:33:53.452342 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:53.452298    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:33:53.688938 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:53.688895    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:33:53.880047 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:53.879972    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:33:54.046333 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:54.046293    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:54.046333 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:54.046304    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:54.046560 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:54.046441    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:33:54.046560 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:54.046481    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:54.046665 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:54.046560    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:33:54.046722 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:54.046668    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:33:54.061616 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:54.061585    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:33:54.958067 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:54.957986    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:33:55.191748 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:55.191716    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:33:55.776052 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:55.776016    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:33:56.046835 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:56.046772    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:56.047279 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:56.046875    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:33:56.047279 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:56.046986    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:56.047279 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:56.046993    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:56.047279 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:56.047096    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:33:56.047279 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:56.047196    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:33:56.585931 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:56.585898    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:33:56.775837 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:56.775796    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:33:57.252943 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:57.252891    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:33:58.046169 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:58.046139    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:58.046340 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:58.046173    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:58.046340 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:58.046236    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:33:58.046417 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:58.046345    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:33:58.046450 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:58.046411    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:58.046526 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:58.046493    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:33:58.961311 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:58.961279    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:33:58.985862 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:58.985836    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs\") pod \"network-metrics-daemon-h5nm6\" (UID: \"08a8e709-63af-4d71-b084-cf533ea620e5\") " pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:33:58.986013 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:58.985883    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:33:58.986013 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:58.986003    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:58.986105 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:58.986024    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:58.986105 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:58.986053    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs podName:08a8e709-63af-4d71-b084-cf533ea620e5 nodeName:}" failed. No retries permitted until 2026-10-30 18:34:30.986040709 +0000 UTC m=+65.422304817 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs") pod "network-metrics-daemon-h5nm6" (UID: "08a8e709-63af-4d71-b084-cf533ea620e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:58.986105 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:58.986072    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret podName:6300215e-2302-47ed-bf47-05a165277c08 nodeName:}" failed. No retries permitted until 2026-10-30 18:34:30.986059763 +0000 UTC m=+65.422323872 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret") pod "global-pull-secret-syncer-gfs79" (UID: "6300215e-2302-47ed-bf47-05a165277c08") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:59.086114 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:59.086087    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdl78\" (UniqueName: \"kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78\") pod \"network-check-target-5gvjh\" (UID: \"a864943c-d7eb-4486-99a6-7961573b57d4\") " pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:33:59.086225 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:59.086208    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:33:59.086225 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:59.086224    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:33:59.086307 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:59.086233    2577 projected.go:196] Error preparing data for projected volume kube-api-access-zdl78 for pod openshift-network-diagnostics/network-check-target-5gvjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:59.086307 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:33:59.086273    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78 podName:a864943c-d7eb-4486-99a6-7961573b57d4 nodeName:}" failed. No retries permitted until 2026-10-30 18:34:31.086262174 +0000 UTC m=+65.522526287 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zdl78" (UniqueName: "kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78") pod "network-check-target-5gvjh" (UID: "a864943c-d7eb-4486-99a6-7961573b57d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:59.742134 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:33:59.742108    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nr7xl_d3fa116f-e372-4a36-9a57-4832dc1440b3/dns-node-resolver/0.log"
Oct 30 18:34:00.048890 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:00.048861    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:34:00.049358 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:00.048980    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:34:00.049358 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:00.048992    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:34:00.049358 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:00.049118    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:34:00.049358 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:00.049171    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:34:00.049358 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:00.049252    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:34:00.163217 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:00.163168    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:00.651560 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:00.651531    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8kldf_7bfc481b-9824-43c5-acf0-436641114b03/node-ca/0.log"
Oct 30 18:34:01.992963 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:01.992912    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:02.046139 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:02.046108    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:34:02.046139 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:02.046132    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:34:02.046323 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:02.046243    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:34:02.046323 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:02.046281    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:34:02.046388 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:02.046338    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:34:02.046420 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:02.046385    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:34:02.166114 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:02.166020    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:02.554623 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:02.554584    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:02.784620 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:02.784583    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:03.126520 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:03.126485    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:03.301090 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:03.301051    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:03.474828 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:03.474751    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:03.960921 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:03.960884    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:04.048515 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:04.048483    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:34:04.048515 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:04.048496    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:34:04.048515 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:04.048496    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:34:04.048803 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:04.048590    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:34:04.048803 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:04.048650    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:34:04.048803 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:04.048718    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:34:04.201395 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:04.201362    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:04.381148 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:04.381119    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:04.582559 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:04.582528    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:04.758687 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:04.758602    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:05.126391 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:05.126351    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:05.326540 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:05.326506    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:05.518066 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:05.517977    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:05.966021 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:05.965991    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:06.046934 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:06.046907    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:34:06.047099 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:06.046983    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:34:06.047099 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:06.047049    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:34:06.047171 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:06.047138    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:34:06.047252 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:06.047233    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:34:06.047403 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:06.047379    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:34:06.188291 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:06.188257    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:06.370212 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:06.370167    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:06.550463 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:06.550426    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:06.727563 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:06.727473    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:06.918343 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:06.918296    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:07.342578 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:07.342547    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:07.949455 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:07.949423    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:08.046476 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:08.046443    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:34:08.046476 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:08.046460    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:34:08.046719 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:08.046457    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:34:08.046719 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:08.046556    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:34:08.046719 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:08.046622    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:34:08.046719 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:08.046681    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:34:08.171730 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:08.171697    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:08.344300 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:08.344267    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:08.775538 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:08.775466    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:09.551350 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:09.551298    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:09.941210 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:09.941175    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:10.046203 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:10.046174    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:34:10.046203 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:10.046187    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:34:10.046439 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:10.046194    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:34:10.046439 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:10.046284    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:34:10.046439 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:10.046353    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:34:10.046592 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:10.046447    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:34:10.150964 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:10.150915    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:10.341591 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:10.341554    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:10.740985 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:10.740884    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:11.148868 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:11.148838    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:11.341131 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:11.341094    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:11.554619 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:11.554591    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:11.733190 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:11.733150    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:12.046226 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:12.046142    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:34:12.046594 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:12.046142    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:34:12.046594 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:12.046261    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:34:12.046594 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:12.046353    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:34:12.046594 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:12.046158    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:34:12.046594 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:12.046453    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:34:12.375305 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:12.375267    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:12.551364 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:12.551331    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:12.727895 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:12.727794    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:13.393118 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:13.393053    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:14.046476 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:14.046427    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:34:14.046476 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:14.046452    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:34:14.046689 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:14.046546    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:34:14.046689 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:14.046585    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:34:14.046766 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:14.046683    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:34:14.046812 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:14.046758    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:34:15.777542 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:15.777497    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:15.955016 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:15.954973    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:16.047221 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:16.047142    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:34:16.047383 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:16.047239    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:34:16.047383 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:16.047269    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h5nm6" podUID="08a8e709-63af-4d71-b084-cf533ea620e5"
Oct 30 18:34:16.047383 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:16.047268    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:34:16.047383 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:16.047373    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5gvjh" podUID="a864943c-d7eb-4486-99a6-7961573b57d4"
Oct 30 18:34:16.047667 ip-10-0-133-80 kubenswrapper[2577]: E1030 18:34:16.047439    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gfs79" podUID="6300215e-2302-47ed-bf47-05a165277c08"
Oct 30 18:34:16.920037 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:16.919997    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:17.119675 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:17.119618    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:17.394780 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:17.394747    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:17.570491 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:17.570465    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:17.857041 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:17.857018    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeReady"
Oct 30 18:34:17.857238 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:17.857153    2577 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 18:34:17.986639 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:17.986610    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-klcg2"]
Oct 30 18:34:17.988388 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:17.988366    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:18.014255 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.014232    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jdtqn"]
Oct 30 18:34:18.014747 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.014728    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ae0bc76-971d-4020-972f-6fb355e71aa0-crio-socket\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:18.014787 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.014766    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ae0bc76-971d-4020-972f-6fb355e71aa0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:18.014823 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.014795    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ae0bc76-971d-4020-972f-6fb355e71aa0-data-volume\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:18.014823 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.014809    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5djbl\" (UniqueName: \"kubernetes.io/projected/2ae0bc76-971d-4020-972f-6fb355e71aa0-kube-api-access-5djbl\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:18.014933 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.014918    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ae0bc76-971d-4020-972f-6fb355e71aa0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:18.015705 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.015692    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:18.023749 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.023729    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-klcg2"]
Oct 30 18:34:18.039572 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.039544    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jdtqn"]
Oct 30 18:34:18.041790 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.041771    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 18:34:18.041790 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.041779    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 18:34:18.041972 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.041822    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 18:34:18.041972 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.041915    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 18:34:18.042076 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.042005    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k76xp\""
Oct 30 18:34:18.046223 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.046206    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:34:18.046304 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.046229    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:34:18.046304 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.046290    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:34:18.064377 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.064354    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-pxlmt"]
Oct 30 18:34:18.065736 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.065723    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pxlmt"
Oct 30 18:34:18.115408 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.115333    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d22b0cdb-141d-4710-9f1f-90797d952a54-tmp-dir\") pod \"dns-default-jdtqn\" (UID: \"d22b0cdb-141d-4710-9f1f-90797d952a54\") " pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:18.115408 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.115395    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ae0bc76-971d-4020-972f-6fb355e71aa0-data-volume\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:18.115591 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.115441    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5djbl\" (UniqueName: \"kubernetes.io/projected/2ae0bc76-971d-4020-972f-6fb355e71aa0-kube-api-access-5djbl\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:18.115591 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.115480    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1538afd4-e5de-4372-aa9c-895efc60d643-cert\") pod \"ingress-canary-pxlmt\" (UID: \"1538afd4-e5de-4372-aa9c-895efc60d643\") " pod="openshift-ingress-canary/ingress-canary-pxlmt"
Oct 30 18:34:18.115591 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.115538    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ae0bc76-971d-4020-972f-6fb355e71aa0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:18.115729 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.115593    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srphr\" (UniqueName: \"kubernetes.io/projected/d22b0cdb-141d-4710-9f1f-90797d952a54-kube-api-access-srphr\") pod \"dns-default-jdtqn\" (UID: \"d22b0cdb-141d-4710-9f1f-90797d952a54\") " pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:18.115729 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.115666    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d22b0cdb-141d-4710-9f1f-90797d952a54-config-volume\") pod \"dns-default-jdtqn\" (UID: \"d22b0cdb-141d-4710-9f1f-90797d952a54\") " pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:18.115823 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.115761    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ae0bc76-971d-4020-972f-6fb355e71aa0-data-volume\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:18.115870 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.115826    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d22b0cdb-141d-4710-9f1f-90797d952a54-metrics-tls\") pod \"dns-default-jdtqn\" (UID: \"d22b0cdb-141d-4710-9f1f-90797d952a54\") " pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:18.115907 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.115874    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbhr8\" (UniqueName: \"kubernetes.io/projected/1538afd4-e5de-4372-aa9c-895efc60d643-kube-api-access-sbhr8\") pod \"ingress-canary-pxlmt\" (UID: \"1538afd4-e5de-4372-aa9c-895efc60d643\") " pod="openshift-ingress-canary/ingress-canary-pxlmt"
Oct 30 18:34:18.115988 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.115975    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ae0bc76-971d-4020-972f-6fb355e71aa0-crio-socket\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:18.116046 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.116008    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ae0bc76-971d-4020-972f-6fb355e71aa0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:18.116191 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.116165    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ae0bc76-971d-4020-972f-6fb355e71aa0-crio-socket\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:18.116258 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.116242    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ae0bc76-971d-4020-972f-6fb355e71aa0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:18.116636 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.116615    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:18.119392 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.119376    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ae0bc76-971d-4020-972f-6fb355e71aa0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:18.125847 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.125824    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pxlmt"]
Oct 30 18:34:18.142240 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.142217    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 18:34:18.142448 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.142299    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 18:34:18.142448 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.142397    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 18:34:18.142448 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.142420    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 18:34:18.142656 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.142500    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 18:34:18.142656 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.142550    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-8wmd6\""
Oct 30 18:34:18.142656 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.142571    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 18:34:18.142656 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.142574    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lmrrc\""
Oct 30 18:34:18.142656 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.142602    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 18:34:18.142909 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.142842    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wkpbb\""
Oct 30 18:34:18.142970 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.142916    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 18:34:18.142970 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.142935    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-727z7\""
Oct 30 18:34:18.142970 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.142967    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-sqz88\""
Oct 30 18:34:18.143069 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.142978    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 18:34:18.177573 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.177546    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5djbl\" (UniqueName: \"kubernetes.io/projected/2ae0bc76-971d-4020-972f-6fb355e71aa0-kube-api-access-5djbl\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:18.217282 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.217246    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srphr\" (UniqueName: \"kubernetes.io/projected/d22b0cdb-141d-4710-9f1f-90797d952a54-kube-api-access-srphr\") pod \"dns-default-jdtqn\" (UID: \"d22b0cdb-141d-4710-9f1f-90797d952a54\") " pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:18.217435 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.217301    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d22b0cdb-141d-4710-9f1f-90797d952a54-config-volume\") pod \"dns-default-jdtqn\" (UID: \"d22b0cdb-141d-4710-9f1f-90797d952a54\") " pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:18.217435 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.217410    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d22b0cdb-141d-4710-9f1f-90797d952a54-metrics-tls\") pod \"dns-default-jdtqn\" (UID: \"d22b0cdb-141d-4710-9f1f-90797d952a54\") " pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:18.217532 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.217447    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbhr8\" (UniqueName: \"kubernetes.io/projected/1538afd4-e5de-4372-aa9c-895efc60d643-kube-api-access-sbhr8\") pod \"ingress-canary-pxlmt\" (UID: \"1538afd4-e5de-4372-aa9c-895efc60d643\") " pod="openshift-ingress-canary/ingress-canary-pxlmt"
Oct 30 18:34:18.217532 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.217472    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d22b0cdb-141d-4710-9f1f-90797d952a54-tmp-dir\") pod \"dns-default-jdtqn\" (UID: \"d22b0cdb-141d-4710-9f1f-90797d952a54\") " pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:18.217647 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.217619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1538afd4-e5de-4372-aa9c-895efc60d643-cert\") pod \"ingress-canary-pxlmt\" (UID: \"1538afd4-e5de-4372-aa9c-895efc60d643\") " pod="openshift-ingress-canary/ingress-canary-pxlmt"
Oct 30 18:34:18.217739 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.217721    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d22b0cdb-141d-4710-9f1f-90797d952a54-tmp-dir\") pod \"dns-default-jdtqn\" (UID: \"d22b0cdb-141d-4710-9f1f-90797d952a54\") " pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:18.217817 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.217799    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d22b0cdb-141d-4710-9f1f-90797d952a54-config-volume\") pod \"dns-default-jdtqn\" (UID: \"d22b0cdb-141d-4710-9f1f-90797d952a54\") " pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:18.219742 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.219723    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d22b0cdb-141d-4710-9f1f-90797d952a54-metrics-tls\") pod \"dns-default-jdtqn\" (UID: \"d22b0cdb-141d-4710-9f1f-90797d952a54\") " pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:18.219935 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.219918    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1538afd4-e5de-4372-aa9c-895efc60d643-cert\") pod \"ingress-canary-pxlmt\" (UID: \"1538afd4-e5de-4372-aa9c-895efc60d643\") " pod="openshift-ingress-canary/ingress-canary-pxlmt"
Oct 30 18:34:18.279299 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.279272    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srphr\" (UniqueName: \"kubernetes.io/projected/d22b0cdb-141d-4710-9f1f-90797d952a54-kube-api-access-srphr\") pod \"dns-default-jdtqn\" (UID: \"d22b0cdb-141d-4710-9f1f-90797d952a54\") " pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:18.279419 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.279354    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbhr8\" (UniqueName: \"kubernetes.io/projected/1538afd4-e5de-4372-aa9c-895efc60d643-kube-api-access-sbhr8\") pod \"ingress-canary-pxlmt\" (UID: \"1538afd4-e5de-4372-aa9c-895efc60d643\") " pod="openshift-ingress-canary/ingress-canary-pxlmt"
Oct 30 18:34:18.288025 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.288001    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:18.299779 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.299764    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:18.325296 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.325267    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:18.374009 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.373874    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pxlmt"
Oct 30 18:34:18.702551 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.702152    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-klcg2"]
Oct 30 18:34:18.741803 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.741773    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jdtqn"]
Oct 30 18:34:18.745979 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:34:18.745927    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd22b0cdb_141d_4710_9f1f_90797d952a54.slice/crio-1cb10ae0ef4e8e9ef9894188e1ce202d8ac2cfdd80afa785ee39fd1bab7d5265 WatchSource:0}: Error finding container 1cb10ae0ef4e8e9ef9894188e1ce202d8ac2cfdd80afa785ee39fd1bab7d5265: Status 404 returned error can't find the container with id 1cb10ae0ef4e8e9ef9894188e1ce202d8ac2cfdd80afa785ee39fd1bab7d5265
Oct 30 18:34:18.754186 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:18.754160    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pxlmt"]
Oct 30 18:34:18.769809 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:34:18.769786    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1538afd4_e5de_4372_aa9c_895efc60d643.slice/crio-2b147797072e843fa03d0d71237c9e289b35e26cebcdd81b9a3b7ff26388ca13 WatchSource:0}: Error finding container 2b147797072e843fa03d0d71237c9e289b35e26cebcdd81b9a3b7ff26388ca13: Status 404 returned error can't find the container with id 2b147797072e843fa03d0d71237c9e289b35e26cebcdd81b9a3b7ff26388ca13
Oct 30 18:34:19.345909 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:19.345875    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jdtqn" event={"ID":"d22b0cdb-141d-4710-9f1f-90797d952a54","Type":"ContainerStarted","Data":"1cb10ae0ef4e8e9ef9894188e1ce202d8ac2cfdd80afa785ee39fd1bab7d5265"}
Oct 30 18:34:19.347369 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:19.347346    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-klcg2" event={"ID":"2ae0bc76-971d-4020-972f-6fb355e71aa0","Type":"ContainerStarted","Data":"8d59c1434df3899af44e9ec5b0295ea26fb5fd51a5aa01f277ae1a620e85a990"}
Oct 30 18:34:19.347476 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:19.347397    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-klcg2" event={"ID":"2ae0bc76-971d-4020-972f-6fb355e71aa0","Type":"ContainerStarted","Data":"55118ad4b9c2f2293db93c7fd1ee53ba589d426955efe3c0aca30e77459d24c9"}
Oct 30 18:34:19.348435 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:19.348418    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pxlmt" event={"ID":"1538afd4-e5de-4372-aa9c-895efc60d643","Type":"ContainerStarted","Data":"2b147797072e843fa03d0d71237c9e289b35e26cebcdd81b9a3b7ff26388ca13"}
Oct 30 18:34:20.353820 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:20.353785    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-klcg2" event={"ID":"2ae0bc76-971d-4020-972f-6fb355e71aa0","Type":"ContainerStarted","Data":"4d23d8685b3acbfe0c28fcf88488a9f643ebb5456f9af97df810dc93e35592c1"}
Oct 30 18:34:21.359339 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:21.359260    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jdtqn" event={"ID":"d22b0cdb-141d-4710-9f1f-90797d952a54","Type":"ContainerStarted","Data":"5b2074734bc126a0ceb615f519d821e2c8261ca18d79834a0050394bb4b8a4fe"}
Oct 30 18:34:21.359339 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:21.359327    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jdtqn" event={"ID":"d22b0cdb-141d-4710-9f1f-90797d952a54","Type":"ContainerStarted","Data":"eac82b721ea9c414bba04f8845669ead6b4893cee061e9c931735f7b81f52c81"}
Oct 30 18:34:21.360052 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:21.360033    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:21.502465 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:21.502429    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:21.534884 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:21.534822    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-jdtqn" podStartSLOduration=2.192831759 podStartE2EDuration="4.534801849s" totalImagesPullingTime="2.34197009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:34:21.486160148 +0000 UTC m=+55.922424280" watchObservedRunningTime="2026-10-30 18:34:21.534801849 +0000 UTC m=+55.971065981"
Oct 30 18:34:21.730194 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:21.730107    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:21.881602 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:21.881578    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"]
Oct 30 18:34:21.884199 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:21.884179    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"
Oct 30 18:34:21.905861 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:21.905838    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:21.940567 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:21.940546    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:34:21.965685 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:21.965654    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-bmxnr\" (UID: \"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"
Oct 30 18:34:21.965805 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:21.965748    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d8jv\" (UniqueName: \"kubernetes.io/projected/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-kube-api-access-7d8jv\") pod \"prometheus-operator-9cd89b74c-bmxnr\" (UID: \"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"
Oct 30 18:34:21.965860 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:21.965813    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-bmxnr\" (UID: \"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"
Oct 30 18:34:21.965860 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:21.965842    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-bmxnr\" (UID: \"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"
Oct 30 18:34:21.969244 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:21.969216    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 18:34:21.970194 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:21.970164    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"]
Oct 30 18:34:21.991383 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:21.991331    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 18:34:21.991383 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:21.990726    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 18:34:21.991713 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:21.991694    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-gg67k\""
Oct 30 18:34:21.992806 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:21.992789    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:34:22.066574 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:22.066544    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-bmxnr\" (UID: \"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"
Oct 30 18:34:22.066743 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:22.066617    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-bmxnr\" (UID: \"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"
Oct 30 18:34:22.066743 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:22.066647    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d8jv\" (UniqueName: \"kubernetes.io/projected/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-kube-api-access-7d8jv\") pod \"prometheus-operator-9cd89b74c-bmxnr\" (UID: \"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"
Oct 30 18:34:22.066743 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:22.066696    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-bmxnr\" (UID: \"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"
Oct 30 18:34:22.067692 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:22.067668    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-bmxnr\" (UID: \"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"
Oct 30 18:34:22.069453 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:22.069427    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-bmxnr\" (UID: \"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"
Oct 30 18:34:22.072423 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:22.072396    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-bmxnr\" (UID: \"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"
Oct 30 18:34:22.129457 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:22.129424    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d8jv\" (UniqueName: \"kubernetes.io/projected/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-kube-api-access-7d8jv\") pod \"prometheus-operator-9cd89b74c-bmxnr\" (UID: \"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"
Oct 30 18:34:22.209566 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:22.209544    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"
Oct 30 18:34:22.363655 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:22.363621    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pxlmt" event={"ID":"1538afd4-e5de-4372-aa9c-895efc60d643","Type":"ContainerStarted","Data":"7a01a3395b2bfd900126a7566daaef13bce7ca0532eaab01fbc53ce36ca6fc0e"}
Oct 30 18:34:22.365962 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:22.365922    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-klcg2" event={"ID":"2ae0bc76-971d-4020-972f-6fb355e71aa0","Type":"ContainerStarted","Data":"11aded59a1e2ae0263b017d6a2be5fa03aed53ce615daa688a73d47d546e88a7"}
Oct 30 18:34:22.481560 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:22.481515    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-pxlmt" podStartSLOduration=1.062253761 podStartE2EDuration="4.481501664s" totalImagesPullingTime="3.419247903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:34:22.481188386 +0000 UTC m=+56.917452517" watchObservedRunningTime="2026-10-30 18:34:22.481501664 +0000 UTC m=+56.917765793"
Oct 30 18:34:22.602074 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:22.602018    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-klcg2" podStartSLOduration=2.6218831910000002 podStartE2EDuration="5.602003473s" totalImagesPullingTime="2.980120282s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:34:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:34:22.601307336 +0000 UTC m=+57.037571467" watchObservedRunningTime="2026-10-30 18:34:22.602003473 +0000 UTC m=+57.038267602"
Oct 30 18:34:22.637078 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:22.637051    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"]
Oct 30 18:34:22.654647 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:34:22.654621    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a4a0b8a_8f26_449f_9f6e_3b91c69bfb84.slice/crio-462aa255cefeccaa7502195b87d7c8ef1d8bef34ec5e20ba24d5e2703aec945d WatchSource:0}: Error finding container 462aa255cefeccaa7502195b87d7c8ef1d8bef34ec5e20ba24d5e2703aec945d: Status 404 returned error can't find the container with id 462aa255cefeccaa7502195b87d7c8ef1d8bef34ec5e20ba24d5e2703aec945d
Oct 30 18:34:23.302399 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:23.302373    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:23.369527 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:23.369490    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr" event={"ID":"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84","Type":"ContainerStarted","Data":"462aa255cefeccaa7502195b87d7c8ef1d8bef34ec5e20ba24d5e2703aec945d"}
Oct 30 18:34:24.143358 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.143330    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-ff87dfcf9-5kf5h"]
Oct 30 18:34:24.145254 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.145221    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.182813 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.182789    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-config\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.182934 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.182820    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-service-ca\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.182934 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.182839    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-oauth-config\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.182934 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.182864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-serving-cert\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.183096 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.182981    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4e65089-4bdd-431f-b338-1596fbb4b02b-tmp\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.183096 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.183010    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-oauth-serving-cert\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.183096 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.183034    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlwnx\" (UniqueName: \"kubernetes.io/projected/d4e65089-4bdd-431f-b338-1596fbb4b02b-kube-api-access-tlwnx\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.205715 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.205691    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-gxn6m\""
Oct 30 18:34:24.216882 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.216838    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 18:34:24.217118 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.217077    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 18:34:24.217272 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.217171    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 18:34:24.256823 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.256749    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 18:34:24.256823 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.256750    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 18:34:24.256823 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.256826    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 18:34:24.267309 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.267285    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 18:34:24.268316 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.268293    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-ff87dfcf9-5kf5h"]
Oct 30 18:34:24.283711 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.283690    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-serving-cert\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.283815 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.283726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4e65089-4bdd-431f-b338-1596fbb4b02b-tmp\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.283815 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.283747    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-oauth-serving-cert\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.283911 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.283857    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlwnx\" (UniqueName: \"kubernetes.io/projected/d4e65089-4bdd-431f-b338-1596fbb4b02b-kube-api-access-tlwnx\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.283987 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.283920    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-config\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.283987 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.283961    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-service-ca\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.284095 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.283995    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-oauth-config\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.284258 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.284235    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4e65089-4bdd-431f-b338-1596fbb4b02b-tmp\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.284551 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.284526    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-oauth-serving-cert\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.284647 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.284583    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-config\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.284744 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.284726    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-service-ca\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.286336 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.286321    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-oauth-config\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.286585 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.286566    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-serving-cert\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.348334 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.347116    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlwnx\" (UniqueName: \"kubernetes.io/projected/d4e65089-4bdd-431f-b338-1596fbb4b02b-kube-api-access-tlwnx\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.373999 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.373974    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr" event={"ID":"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84","Type":"ContainerStarted","Data":"2c43469d6fbfe6a345f6575be7f921888d3dc9264a7a624647e257ffc20be683"}
Oct 30 18:34:24.456666 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.456630    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:24.824614 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:24.824584    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-ff87dfcf9-5kf5h"]
Oct 30 18:34:24.855396 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:34:24.855373    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4e65089_4bdd_431f_b338_1596fbb4b02b.slice/crio-43cd4cc8435d93fd0c9c87d3f00169ba3775f56ea3e2eafdadb56403d61fc493 WatchSource:0}: Error finding container 43cd4cc8435d93fd0c9c87d3f00169ba3775f56ea3e2eafdadb56403d61fc493: Status 404 returned error can't find the container with id 43cd4cc8435d93fd0c9c87d3f00169ba3775f56ea3e2eafdadb56403d61fc493
Oct 30 18:34:25.377705 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:25.377669    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ff87dfcf9-5kf5h" event={"ID":"d4e65089-4bdd-431f-b338-1596fbb4b02b","Type":"ContainerStarted","Data":"43cd4cc8435d93fd0c9c87d3f00169ba3775f56ea3e2eafdadb56403d61fc493"}
Oct 30 18:34:25.379362 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:25.379339    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr" event={"ID":"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84","Type":"ContainerStarted","Data":"70d74cc34065fb1f17136373d554e4046bec026f0d99052c24e8b81b9383bc20"}
Oct 30 18:34:25.499348 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:25.499302    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr" podStartSLOduration=2.8591531310000002 podStartE2EDuration="4.499285572s" totalImagesPullingTime="1.640132441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:34:25.498628704 +0000 UTC m=+59.934892836" watchObservedRunningTime="2026-10-30 18:34:25.499285572 +0000 UTC m=+59.935549703"
Oct 30 18:34:27.047682 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:27.047590    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:27.683796 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:27.683763    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:27.908079 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:27.908033    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:28.158136 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.158098    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:28.320595 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.320569    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"]
Oct 30 18:34:28.322299 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.322285    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"
Oct 30 18:34:28.349284 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.349260    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:28.375684 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.375659    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 18:34:28.390853 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.390817    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ff87dfcf9-5kf5h" event={"ID":"d4e65089-4bdd-431f-b338-1596fbb4b02b","Type":"ContainerStarted","Data":"a353b018084a1b0711d5cd1443b435040145652aab2e4f4cb5ce7a1e6ceabe9c"}
Oct 30 18:34:28.414554 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.414526    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9c43ecd-b447-4e07-95cf-01523d71dda9-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-g4h9b\" (UID: \"a9c43ecd-b447-4e07-95cf-01523d71dda9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"
Oct 30 18:34:28.414687 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.414563    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a9c43ecd-b447-4e07-95cf-01523d71dda9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-g4h9b\" (UID: \"a9c43ecd-b447-4e07-95cf-01523d71dda9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"
Oct 30 18:34:28.414687 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.414597    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9c43ecd-b447-4e07-95cf-01523d71dda9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-g4h9b\" (UID: \"a9c43ecd-b447-4e07-95cf-01523d71dda9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"
Oct 30 18:34:28.414759 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.414737    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2r2b\" (UniqueName: \"kubernetes.io/projected/a9c43ecd-b447-4e07-95cf-01523d71dda9-kube-api-access-l2r2b\") pod \"openshift-state-metrics-7bd7489957-g4h9b\" (UID: \"a9c43ecd-b447-4e07-95cf-01523d71dda9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"
Oct 30 18:34:28.425994 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.425942    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-rwjgn\""
Oct 30 18:34:28.426055 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.425969    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 18:34:28.427240 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.427218    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"]
Oct 30 18:34:28.427864 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.427846    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"]
Oct 30 18:34:28.429767 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.429755    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:28.478005 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.477981    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-x9dhg"]
Oct 30 18:34:28.479782 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.479766    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"]
Oct 30 18:34:28.479863 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.479853    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.483360 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.483311    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 18:34:28.488654 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.488634    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 18:34:28.488654 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.488649    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 18:34:28.488821 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.488701    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-b5h66\""
Oct 30 18:34:28.515079 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.515043    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:28.515079 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.515087    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42nvt\" (UniqueName: \"kubernetes.io/projected/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-api-access-42nvt\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:28.515285 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.515223    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e4104cc2-a8a5-4985-83cd-e593bf12a68f-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:28.515285 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.515265    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:28.515381 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.515339    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:28.515449 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.515434    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2r2b\" (UniqueName: \"kubernetes.io/projected/a9c43ecd-b447-4e07-95cf-01523d71dda9-kube-api-access-l2r2b\") pod \"openshift-state-metrics-7bd7489957-g4h9b\" (UID: \"a9c43ecd-b447-4e07-95cf-01523d71dda9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"
Oct 30 18:34:28.515506 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.515459    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4104cc2-a8a5-4985-83cd-e593bf12a68f-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:28.515506 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.515495    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9c43ecd-b447-4e07-95cf-01523d71dda9-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-g4h9b\" (UID: \"a9c43ecd-b447-4e07-95cf-01523d71dda9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"
Oct 30 18:34:28.515594 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.515563    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a9c43ecd-b447-4e07-95cf-01523d71dda9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-g4h9b\" (UID: \"a9c43ecd-b447-4e07-95cf-01523d71dda9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"
Oct 30 18:34:28.515637 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.515599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9c43ecd-b447-4e07-95cf-01523d71dda9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-g4h9b\" (UID: \"a9c43ecd-b447-4e07-95cf-01523d71dda9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"
Oct 30 18:34:28.516401 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.516328    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9c43ecd-b447-4e07-95cf-01523d71dda9-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-g4h9b\" (UID: \"a9c43ecd-b447-4e07-95cf-01523d71dda9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"
Oct 30 18:34:28.518122 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.518102    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a9c43ecd-b447-4e07-95cf-01523d71dda9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-g4h9b\" (UID: \"a9c43ecd-b447-4e07-95cf-01523d71dda9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"
Oct 30 18:34:28.518207 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.518188    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9c43ecd-b447-4e07-95cf-01523d71dda9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-g4h9b\" (UID: \"a9c43ecd-b447-4e07-95cf-01523d71dda9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"
Oct 30 18:34:28.531432 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.531408    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:28.577675 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.577624    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2r2b\" (UniqueName: \"kubernetes.io/projected/a9c43ecd-b447-4e07-95cf-01523d71dda9-kube-api-access-l2r2b\") pod \"openshift-state-metrics-7bd7489957-g4h9b\" (UID: \"a9c43ecd-b447-4e07-95cf-01523d71dda9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"
Oct 30 18:34:28.583202 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.583182    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 18:34:28.583202 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.583195    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 18:34:28.583571 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.583552    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tc629\""
Oct 30 18:34:28.583651 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.583622    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 18:34:28.616375 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.616339    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8876fbb5-cdb6-4755-9122-8640d5c86560-root\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.616476 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.616384    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:28.616476 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.616412    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42nvt\" (UniqueName: \"kubernetes.io/projected/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-api-access-42nvt\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:28.616476 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.616469    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km4hl\" (UniqueName: \"kubernetes.io/projected/8876fbb5-cdb6-4755-9122-8640d5c86560-kube-api-access-km4hl\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.616599 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.616528    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e4104cc2-a8a5-4985-83cd-e593bf12a68f-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:28.616599 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.616556    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.616599 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.616585    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:28.616741 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.616628    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-wtmp\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.616795 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.616772    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:28.616849 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.616822    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-accelerators-collector-config\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.616902 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.616853    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-tls\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.616902 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.616884    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8876fbb5-cdb6-4755-9122-8640d5c86560-metrics-client-ca\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.617016 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.616941    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-textfile\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.617064 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.617029    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4104cc2-a8a5-4985-83cd-e593bf12a68f-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:28.617064 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.617035    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e4104cc2-a8a5-4985-83cd-e593bf12a68f-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:28.617136 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.617065    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8876fbb5-cdb6-4755-9122-8640d5c86560-sys\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.617545 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.617530    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4104cc2-a8a5-4985-83cd-e593bf12a68f-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:28.617650 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.617628    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:28.619245 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.619223    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:28.619450 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.619431    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:28.634119 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.634101    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"
Oct 30 18:34:28.677670 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.677612    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-ff87dfcf9-5kf5h" podStartSLOduration=1.535706739 podStartE2EDuration="4.677594474s" totalImagesPullingTime="3.141887735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:34:28.676753831 +0000 UTC m=+63.113017961" watchObservedRunningTime="2026-10-30 18:34:28.677594474 +0000 UTC m=+63.113858604"
Oct 30 18:34:28.680588 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.680478    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42nvt\" (UniqueName: \"kubernetes.io/projected/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-api-access-42nvt\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:28.709797 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.709757    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:28.717702 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.717680    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8876fbb5-cdb6-4755-9122-8640d5c86560-sys\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.717826 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.717726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8876fbb5-cdb6-4755-9122-8640d5c86560-root\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.717826 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.717751    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km4hl\" (UniqueName: \"kubernetes.io/projected/8876fbb5-cdb6-4755-9122-8640d5c86560-kube-api-access-km4hl\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.717826 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.717785    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.717826 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.717804    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-wtmp\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.717826 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.717799    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8876fbb5-cdb6-4755-9122-8640d5c86560-sys\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.717826 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.717824    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-accelerators-collector-config\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.718128 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.717800    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8876fbb5-cdb6-4755-9122-8640d5c86560-root\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.718128 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.717848    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-tls\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.718128 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.717877    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8876fbb5-cdb6-4755-9122-8640d5c86560-metrics-client-ca\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.718128 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.717898    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-textfile\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.718128 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.717977    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-wtmp\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.718306 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.718288    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-textfile\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.718591 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.718533    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-accelerators-collector-config\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.718591 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.718536    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8876fbb5-cdb6-4755-9122-8640d5c86560-metrics-client-ca\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.720089 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.720068    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.720198 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.720182    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-tls\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.741190 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.741169    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:28.785856 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.785644    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-km4hl\" (UniqueName: \"kubernetes.io/projected/8876fbb5-cdb6-4755-9122-8640d5c86560-kube-api-access-km4hl\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.791459 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.791244    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:28.801771 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:34:28.801743    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8876fbb5_cdb6_4755_9122_8640d5c86560.slice/crio-79a1052e4b6242529ce9541bfff5f9b476b8f9750cf6e6de0732ccfc3b54ceb3 WatchSource:0}: Error finding container 79a1052e4b6242529ce9541bfff5f9b476b8f9750cf6e6de0732ccfc3b54ceb3: Status 404 returned error can't find the container with id 79a1052e4b6242529ce9541bfff5f9b476b8f9750cf6e6de0732ccfc3b54ceb3
Oct 30 18:34:28.885622 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:28.885544    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:29.001158 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:29.000931    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"]
Oct 30 18:34:29.003365 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:34:29.003336    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9c43ecd_b447_4e07_95cf_01523d71dda9.slice/crio-0590af5ae74b4a40ae48bbd883a43b993ae845015c639ef0b857b46f368291ea WatchSource:0}: Error finding container 0590af5ae74b4a40ae48bbd883a43b993ae845015c639ef0b857b46f368291ea: Status 404 returned error can't find the container with id 0590af5ae74b4a40ae48bbd883a43b993ae845015c639ef0b857b46f368291ea
Oct 30 18:34:29.111033 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:29.111008    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"]
Oct 30 18:34:29.114329 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:34:29.114303    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4104cc2_a8a5_4985_83cd_e593bf12a68f.slice/crio-514e0a6bd3f70c8d6839d7c8b54472ea7601330ae334354af1c1c1f3293f031c WatchSource:0}: Error finding container 514e0a6bd3f70c8d6839d7c8b54472ea7601330ae334354af1c1c1f3293f031c: Status 404 returned error can't find the container with id 514e0a6bd3f70c8d6839d7c8b54472ea7601330ae334354af1c1c1f3293f031c
Oct 30 18:34:29.215771 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:29.215695    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:29.393581 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:29.393535    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:29.395276 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:29.395246    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b" event={"ID":"a9c43ecd-b447-4e07-95cf-01523d71dda9","Type":"ContainerStarted","Data":"f7a0ebeaae7a622a9061fb78cb0da26f80bb122add77c275cf7fbf4947f8c071"}
Oct 30 18:34:29.395418 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:29.395323    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b" event={"ID":"a9c43ecd-b447-4e07-95cf-01523d71dda9","Type":"ContainerStarted","Data":"77d7a87bd4a38db703c511c8eba349f7e7e587dbed73bb96e73bcface4598a41"}
Oct 30 18:34:29.395418 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:29.395354    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b" event={"ID":"a9c43ecd-b447-4e07-95cf-01523d71dda9","Type":"ContainerStarted","Data":"0590af5ae74b4a40ae48bbd883a43b993ae845015c639ef0b857b46f368291ea"}
Oct 30 18:34:29.396500 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:29.396482    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x9dhg" event={"ID":"8876fbb5-cdb6-4755-9122-8640d5c86560","Type":"ContainerStarted","Data":"79a1052e4b6242529ce9541bfff5f9b476b8f9750cf6e6de0732ccfc3b54ceb3"}
Oct 30 18:34:29.397541 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:29.397513    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv" event={"ID":"e4104cc2-a8a5-4985-83cd-e593bf12a68f","Type":"ContainerStarted","Data":"514e0a6bd3f70c8d6839d7c8b54472ea7601330ae334354af1c1c1f3293f031c"}
Oct 30 18:34:29.827499 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:29.827459    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:30.046201 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.046163    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:30.220745 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.220661    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:30.404968 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.404920    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x9dhg" event={"ID":"8876fbb5-cdb6-4755-9122-8640d5c86560","Type":"ContainerStarted","Data":"6ae1cf197f8b00632c725672f174a89a01ca2aaa0c558a30ecdee9beed00a8d0"}
Oct 30 18:34:30.709523 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.709492    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:30.717494 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.717469    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 18:34:30.720889 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.720866    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.774385 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.774351    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 18:34:30.774525 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.774444    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 18:34:30.774525 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.774442    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 18:34:30.774525 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.774442    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-gcq7x\""
Oct 30 18:34:30.774648 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.774548    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 18:34:30.777977 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.777910    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 18:34:30.777977 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.777921    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 18:34:30.778238 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.777996    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 18:34:30.778238 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.778010    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 18:34:30.829640 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.829562    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 18:34:30.838154 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.838132    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f7ed38b7-0997-4cd8-9da0-41093596a6a0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.838297 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.838160    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-web-config\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.838297 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.838180    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f7ed38b7-0997-4cd8-9da0-41093596a6a0-config-out\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.838297 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.838211    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.838451 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.838345    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7ed38b7-0997-4cd8-9da0-41093596a6a0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.838451 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.838395    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-config-volume\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.838451 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.838426    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.838587 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.838472    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.838587 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.838510    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4bcf\" (UniqueName: \"kubernetes.io/projected/f7ed38b7-0997-4cd8-9da0-41093596a6a0-kube-api-access-d4bcf\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.838587 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.838539    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7ed38b7-0997-4cd8-9da0-41093596a6a0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.838587 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.838575    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.838774 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.838642    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.838774 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.838669    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f7ed38b7-0997-4cd8-9da0-41093596a6a0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.877137 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.877105    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 18:34:30.939097 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.939029    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-config-volume\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.939097 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.939063    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.939097 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.939090    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.939372 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.939113    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4bcf\" (UniqueName: \"kubernetes.io/projected/f7ed38b7-0997-4cd8-9da0-41093596a6a0-kube-api-access-d4bcf\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.939372 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.939233    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7ed38b7-0997-4cd8-9da0-41093596a6a0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.939372 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.939280    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.939372 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.939344    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.939573 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.939373    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f7ed38b7-0997-4cd8-9da0-41093596a6a0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.939573 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.939405    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f7ed38b7-0997-4cd8-9da0-41093596a6a0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.939573 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.939429    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-web-config\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.939573 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.939477    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f7ed38b7-0997-4cd8-9da0-41093596a6a0-config-out\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.939573 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.939506    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.939573 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.939536    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7ed38b7-0997-4cd8-9da0-41093596a6a0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.940669 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.940013    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7ed38b7-0997-4cd8-9da0-41093596a6a0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.940669 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.940560    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7ed38b7-0997-4cd8-9da0-41093596a6a0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.940669 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.940583    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f7ed38b7-0997-4cd8-9da0-41093596a6a0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.942442 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.942327    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-config-volume\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.943109 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.942673    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.943109 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.943068    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.943109 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.943092    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.943299 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.943160    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-web-config\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.943299 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.943213    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.943472 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.943448    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f7ed38b7-0997-4cd8-9da0-41093596a6a0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.943888 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.943864    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:30.944223 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:30.944206    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f7ed38b7-0997-4cd8-9da0-41093596a6a0-config-out\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:31.001117 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.001093    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4bcf\" (UniqueName: \"kubernetes.io/projected/f7ed38b7-0997-4cd8-9da0-41093596a6a0-kube-api-access-d4bcf\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:31.040040 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.040012    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:34:31.040137 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.040066    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs\") pod \"network-metrics-daemon-h5nm6\" (UID: \"08a8e709-63af-4d71-b084-cf533ea620e5\") " pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:34:31.042777 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.042757    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:31.057333 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.057310    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"]
Oct 30 18:34:31.060757 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.060739    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.093477 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.092419    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 18:34:31.093477 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.092681    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 18:34:31.105980 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.103763    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:34:31.108302 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.107492    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs\") pod \"network-metrics-daemon-h5nm6\" (UID: \"08a8e709-63af-4d71-b084-cf533ea620e5\") " pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:34:31.114735 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.114711    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 18:34:31.114837 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.114769    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 18:34:31.114837 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.114783    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 18:34:31.114837 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.114783    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 18:34:31.115007 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.114862    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-8ln3so4ei8qsv\""
Oct 30 18:34:31.115007 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.114899    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-nxqnw\""
Oct 30 18:34:31.115007 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.114917    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 18:34:31.125554 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.125529    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"]
Oct 30 18:34:31.141073 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.141052    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.141179 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.141090    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.141179 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.141140    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.141262 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.141184    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/220393aa-93ee-4113-b373-ecd27198f170-metrics-client-ca\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.141262 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.141204    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-tls\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.141338 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.141310    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/220393aa-93ee-4113-b373-ecd27198f170-tmp\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.141382 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.141345    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.141450 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.141431    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7rqz\" (UniqueName: \"kubernetes.io/projected/220393aa-93ee-4113-b373-ecd27198f170-kube-api-access-j7rqz\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.141547 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.141530    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-grpc-tls\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.141594 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.141578    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdl78\" (UniqueName: \"kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78\") pod \"network-check-target-5gvjh\" (UID: \"a864943c-d7eb-4486-99a6-7961573b57d4\") " pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:34:31.193975 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.193886    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 18:34:31.242729 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.242698    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-grpc-tls\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.243120 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.242734    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.243120 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.242753    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.243120 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.243028    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.243120 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.243062    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/220393aa-93ee-4113-b373-ecd27198f170-metrics-client-ca\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.243120 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.243084    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-tls\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.243120 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.243114    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/220393aa-93ee-4113-b373-ecd27198f170-tmp\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.243444 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.243140    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.243444 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.243175    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7rqz\" (UniqueName: \"kubernetes.io/projected/220393aa-93ee-4113-b373-ecd27198f170-kube-api-access-j7rqz\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.243749 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.243721    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/220393aa-93ee-4113-b373-ecd27198f170-metrics-client-ca\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.243873 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.243857    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/220393aa-93ee-4113-b373-ecd27198f170-tmp\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.245775 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.245730    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-tls\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.245991 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.245971    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.246059 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.246038    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.246142 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.246108    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.246189 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.246157    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.246268 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.246252    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-grpc-tls\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.254093 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.254073    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 18:34:31.264259 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.264230    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdl78\" (UniqueName: \"kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78\") pod \"network-check-target-5gvjh\" (UID: \"a864943c-d7eb-4486-99a6-7961573b57d4\") " pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:34:31.305277 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.305253    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7rqz\" (UniqueName: \"kubernetes.io/projected/220393aa-93ee-4113-b373-ecd27198f170-kube-api-access-j7rqz\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.314318 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.314299    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wkpbb\""
Oct 30 18:34:31.318869 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.318855    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-727z7\""
Oct 30 18:34:31.323004 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.322978    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:34:31.327880 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.327864    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:34:31.386156 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.385538    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:31.409941 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.409912    2577 generic.go:356] "Generic (PLEG): container finished" podID="8876fbb5-cdb6-4755-9122-8640d5c86560" containerID="6ae1cf197f8b00632c725672f174a89a01ca2aaa0c558a30ecdee9beed00a8d0" exitCode=0
Oct 30 18:34:31.410081 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.410046    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x9dhg" event={"ID":"8876fbb5-cdb6-4755-9122-8640d5c86560","Type":"ContainerDied","Data":"6ae1cf197f8b00632c725672f174a89a01ca2aaa0c558a30ecdee9beed00a8d0"}
Oct 30 18:34:31.419332 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.418600    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv" event={"ID":"e4104cc2-a8a5-4985-83cd-e593bf12a68f","Type":"ContainerStarted","Data":"3d16f4981e659919b526319cb73a9ad4ee8e7fde5b8d470b0de625d411e531a5"}
Oct 30 18:34:31.419332 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.418644    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv" event={"ID":"e4104cc2-a8a5-4985-83cd-e593bf12a68f","Type":"ContainerStarted","Data":"1dee3581f9ed34cd05792df087497dd4ca261b5b08230791e09bbc97070d97f7"}
Oct 30 18:34:31.419332 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.418662    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv" event={"ID":"e4104cc2-a8a5-4985-83cd-e593bf12a68f","Type":"ContainerStarted","Data":"1ea267191ddb569c189e67d6bd87eb6a12e8d671a6d68ba2805f32a534a7ec6b"}
Oct 30 18:34:31.421198 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.421162    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b" event={"ID":"a9c43ecd-b447-4e07-95cf-01523d71dda9","Type":"ContainerStarted","Data":"03efb5fdc6fde6598ac8dade019bb4599e4213e393f62c83118af3fd4ffffda1"}
Oct 30 18:34:31.479884 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.479863    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 18:34:31.482113 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:34:31.482084    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7ed38b7_0997_4cd8_9da0_41093596a6a0.slice/crio-b1e40d3592a73c7b18b1a77643b79ec547e87f7468511e27bab38a1ba707246d WatchSource:0}: Error finding container b1e40d3592a73c7b18b1a77643b79ec547e87f7468511e27bab38a1ba707246d: Status 404 returned error can't find the container with id b1e40d3592a73c7b18b1a77643b79ec547e87f7468511e27bab38a1ba707246d
Oct 30 18:34:31.608938 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.608912    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-8wmd6\""
Oct 30 18:34:31.616986 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.616964    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:34:31.652302 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.652250    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b" podStartSLOduration=2.096502666 podStartE2EDuration="3.652235072s" totalImagesPullingTime="1.555732406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:34:31.651713368 +0000 UTC m=+66.087977499" watchObservedRunningTime="2026-10-30 18:34:31.652235072 +0000 UTC m=+66.088499205"
Oct 30 18:34:31.701633 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.698394    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-h5nm6"]
Oct 30 18:34:31.701988 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.701773    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gfs79"]
Oct 30 18:34:31.706396 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:34:31.706373    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08a8e709_63af_4d71_b084_cf533ea620e5.slice/crio-58b00c4b2632f708912066d204eb2308f9064bb14376596a20357463a81951c7 WatchSource:0}: Error finding container 58b00c4b2632f708912066d204eb2308f9064bb14376596a20357463a81951c7: Status 404 returned error can't find the container with id 58b00c4b2632f708912066d204eb2308f9064bb14376596a20357463a81951c7
Oct 30 18:34:31.709298 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:34:31.709283    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6300215e_2302_47ed_bf47_05a165277c08.slice/crio-0391a54f7e105251436e4777f9ef3d0bf9de145bfdf57e6d4791f0e2de41f9c5 WatchSource:0}: Error finding container 0391a54f7e105251436e4777f9ef3d0bf9de145bfdf57e6d4791f0e2de41f9c5: Status 404 returned error can't find the container with id 0391a54f7e105251436e4777f9ef3d0bf9de145bfdf57e6d4791f0e2de41f9c5
Oct 30 18:34:31.772314 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.772267    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv" podStartSLOduration=1.927819132 podStartE2EDuration="3.772255003s" totalImagesPullingTime="1.844435871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:34:31.771086736 +0000 UTC m=+66.207350866" watchObservedRunningTime="2026-10-30 18:34:31.772255003 +0000 UTC m=+66.208519132"
Oct 30 18:34:31.775588 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.775567    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"]
Oct 30 18:34:31.777477 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:34:31.777457    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod220393aa_93ee_4113_b373_ecd27198f170.slice/crio-22955d982cb91e10acdb17253bb5869feb48742405a22e2aebe0946d82f756ba WatchSource:0}: Error finding container 22955d982cb91e10acdb17253bb5869feb48742405a22e2aebe0946d82f756ba: Status 404 returned error can't find the container with id 22955d982cb91e10acdb17253bb5869feb48742405a22e2aebe0946d82f756ba
Oct 30 18:34:31.994558 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:31.994534    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5gvjh"]
Oct 30 18:34:31.996488 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:34:31.996456    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda864943c_d7eb_4486_99a6_7961573b57d4.slice/crio-f1248a0b7920156836ee332eae6b906552c53e8e532641c6e807aa6e3399b40c WatchSource:0}: Error finding container f1248a0b7920156836ee332eae6b906552c53e8e532641c6e807aa6e3399b40c: Status 404 returned error can't find the container with id f1248a0b7920156836ee332eae6b906552c53e8e532641c6e807aa6e3399b40c
Oct 30 18:34:32.372505 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:32.372473    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:32.428455 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:32.428423    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x9dhg" event={"ID":"8876fbb5-cdb6-4755-9122-8640d5c86560","Type":"ContainerStarted","Data":"dc6df55e5dbda68ae64b032b486e2fae26d8b7e9df6704f35a390e897d4f83d4"}
Oct 30 18:34:32.428618 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:32.428480    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x9dhg" event={"ID":"8876fbb5-cdb6-4755-9122-8640d5c86560","Type":"ContainerStarted","Data":"2dd8d6d26882d6d341368dce2c7ba3d3718fb346771d3c190d8fff91133fa3bd"}
Oct 30 18:34:32.429782 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:32.429752    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-h5nm6" event={"ID":"08a8e709-63af-4d71-b084-cf533ea620e5","Type":"ContainerStarted","Data":"58b00c4b2632f708912066d204eb2308f9064bb14376596a20357463a81951c7"}
Oct 30 18:34:32.431003 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:32.430975    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd" event={"ID":"220393aa-93ee-4113-b373-ecd27198f170","Type":"ContainerStarted","Data":"22955d982cb91e10acdb17253bb5869feb48742405a22e2aebe0946d82f756ba"}
Oct 30 18:34:32.432333 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:32.432309    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7ed38b7-0997-4cd8-9da0-41093596a6a0","Type":"ContainerStarted","Data":"b1e40d3592a73c7b18b1a77643b79ec547e87f7468511e27bab38a1ba707246d"}
Oct 30 18:34:32.433726 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:32.433706    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5gvjh" event={"ID":"a864943c-d7eb-4486-99a6-7961573b57d4","Type":"ContainerStarted","Data":"f1248a0b7920156836ee332eae6b906552c53e8e532641c6e807aa6e3399b40c"}
Oct 30 18:34:32.435482 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:32.435460    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gfs79" event={"ID":"6300215e-2302-47ed-bf47-05a165277c08","Type":"ContainerStarted","Data":"0391a54f7e105251436e4777f9ef3d0bf9de145bfdf57e6d4791f0e2de41f9c5"}
Oct 30 18:34:32.528552 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:32.528518    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:32.758706 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:32.758578    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:32.964600 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:32.964564    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:33.160543 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.160372    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:33.174236 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.174097    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-x9dhg" podStartSLOduration=3.64701547 podStartE2EDuration="5.174077881s" totalImagesPullingTime="1.441118421s" totalInitContainerRuntime="85.94399ms" isStatefulPod=true podCreationTimestamp="2026-10-30 18:34:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:34:32.645108603 +0000 UTC m=+67.081372733" watchObservedRunningTime="2026-10-30 18:34:33.174077881 +0000 UTC m=+67.610342014"
Oct 30 18:34:33.174826 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.174797    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j"]
Oct 30 18:34:33.179277 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.178411    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j"
Oct 30 18:34:33.234444 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.233165    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7445978df6-sf44q"]
Oct 30 18:34:33.237370 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.237347    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.242411 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.242206    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 18:34:33.242411 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.242292    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-wd6sn\""
Oct 30 18:34:33.244137 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.244079    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j"]
Oct 30 18:34:33.262486 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.262463    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc7462f0-a45f-487a-99b9-3e89f783cb01-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-cgw6j\" (UID: \"dc7462f0-a45f-487a-99b9-3e89f783cb01\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j"
Oct 30 18:34:33.262610 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.262497    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj846\" (UniqueName: \"kubernetes.io/projected/dc7462f0-a45f-487a-99b9-3e89f783cb01-kube-api-access-vj846\") pod \"monitoring-plugin-77f4f76594-cgw6j\" (UID: \"dc7462f0-a45f-487a-99b9-3e89f783cb01\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j"
Oct 30 18:34:33.291022 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.290996    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 18:34:33.291154 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.291080    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 18:34:33.300309 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.300260    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7445978df6-sf44q"]
Oct 30 18:34:33.313746 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.313717    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 18:34:33.313865 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.313801    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qxrcd\""
Oct 30 18:34:33.313865 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.313819    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 18:34:33.314031 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.313888    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-453p4sl0bkn6n\""
Oct 30 18:34:33.353357 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.353288    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:33.368185 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.368153    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc7462f0-a45f-487a-99b9-3e89f783cb01-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-cgw6j\" (UID: \"dc7462f0-a45f-487a-99b9-3e89f783cb01\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j"
Oct 30 18:34:33.368364 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.368207    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-audit-log\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.368364 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.368253    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj846\" (UniqueName: \"kubernetes.io/projected/dc7462f0-a45f-487a-99b9-3e89f783cb01-kube-api-access-vj846\") pod \"monitoring-plugin-77f4f76594-cgw6j\" (UID: \"dc7462f0-a45f-487a-99b9-3e89f783cb01\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j"
Oct 30 18:34:33.368364 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.368287    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.368364 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.368334    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-secret-metrics-server-client-certs\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.368673 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.368377    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-metrics-server-audit-profiles\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.368673 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.368430    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-client-ca-bundle\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.368673 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.368474    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-secret-metrics-server-tls\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.368673 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.368513    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rlnr\" (UniqueName: \"kubernetes.io/projected/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-kube-api-access-4rlnr\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.378797 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.378767    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc7462f0-a45f-487a-99b9-3e89f783cb01-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-cgw6j\" (UID: \"dc7462f0-a45f-487a-99b9-3e89f783cb01\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j"
Oct 30 18:34:33.434397 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.434369    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj846\" (UniqueName: \"kubernetes.io/projected/dc7462f0-a45f-487a-99b9-3e89f783cb01-kube-api-access-vj846\") pod \"monitoring-plugin-77f4f76594-cgw6j\" (UID: \"dc7462f0-a45f-487a-99b9-3e89f783cb01\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j"
Oct 30 18:34:33.443986 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.443935    2577 generic.go:356] "Generic (PLEG): container finished" podID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerID="62e3c9bf7e576b3b3d45f029b9b4d48e7d2bdf1e1c03a0103679146beaf27a5a" exitCode=0
Oct 30 18:34:33.444199 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.444125    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7ed38b7-0997-4cd8-9da0-41093596a6a0","Type":"ContainerDied","Data":"62e3c9bf7e576b3b3d45f029b9b4d48e7d2bdf1e1c03a0103679146beaf27a5a"}
Oct 30 18:34:33.469333 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.469304    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-audit-log\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.469426 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.469355    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.469426 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.469404    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-secret-metrics-server-client-certs\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.469528 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.469433    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-metrics-server-audit-profiles\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.469528 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.469471    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-client-ca-bundle\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.469528 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.469505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-secret-metrics-server-tls\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.469682 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.469538    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rlnr\" (UniqueName: \"kubernetes.io/projected/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-kube-api-access-4rlnr\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.470249 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.470207    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-audit-log\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.470839 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.470816    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.472129 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.472103    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-metrics-server-audit-profiles\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.474286 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.474246    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-client-ca-bundle\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.474372 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.474322    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-secret-metrics-server-tls\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.484745 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.484723    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-secret-metrics-server-client-certs\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.495641 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.495619    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j"
Oct 30 18:34:33.538215 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.538185    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:33.548682 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.548655    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rlnr\" (UniqueName: \"kubernetes.io/projected/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-kube-api-access-4rlnr\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.563434 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.563406    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:33.907245 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:33.907176    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j"]
Oct 30 18:34:33.912718 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:34:33.912691    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc7462f0_a45f_487a_99b9_3e89f783cb01.slice/crio-e24685c45cd1c1f042e3505248a05f1579704e97d3d9750ca6a6bcd82611622a WatchSource:0}: Error finding container e24685c45cd1c1f042e3505248a05f1579704e97d3d9750ca6a6bcd82611622a: Status 404 returned error can't find the container with id e24685c45cd1c1f042e3505248a05f1579704e97d3d9750ca6a6bcd82611622a
Oct 30 18:34:34.021294 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.021026    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7445978df6-sf44q"]
Oct 30 18:34:34.024495 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:34:34.024435    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf17e11d_1428_47c8_9aea_9f6b7c831eb1.slice/crio-48aec6da0f776fb642caeda9775dc3cb704583591bc8a7d7fe453b8c84663736 WatchSource:0}: Error finding container 48aec6da0f776fb642caeda9775dc3cb704583591bc8a7d7fe453b8c84663736: Status 404 returned error can't find the container with id 48aec6da0f776fb642caeda9775dc3cb704583591bc8a7d7fe453b8c84663736
Oct 30 18:34:34.121285 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.121243    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-59c4fb9dcd-xzxdr"]
Oct 30 18:34:34.124400 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.124328    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.181124 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.181055    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 18:34:34.232643 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.232570    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59c4fb9dcd-xzxdr"]
Oct 30 18:34:34.281332 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.281296    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-console-config\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.281481 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.281363    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/824f84de-51ea-4435-9aee-9fbcde961d41-tmp\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.281481 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.281391    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-trusted-ca-bundle\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.281481 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.281425    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/824f84de-51ea-4435-9aee-9fbcde961d41-console-serving-cert\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.281481 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.281450    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-service-ca\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.281481 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.281476    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/824f84de-51ea-4435-9aee-9fbcde961d41-console-oauth-config\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.281682 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.281502    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-oauth-serving-cert\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.281682 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.281559    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4z99\" (UniqueName: \"kubernetes.io/projected/824f84de-51ea-4435-9aee-9fbcde961d41-kube-api-access-t4z99\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.380225 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.380190    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:34.382919 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.382893    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4z99\" (UniqueName: \"kubernetes.io/projected/824f84de-51ea-4435-9aee-9fbcde961d41-kube-api-access-t4z99\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.383073 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.382926    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-console-config\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.383073 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.383007    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/824f84de-51ea-4435-9aee-9fbcde961d41-tmp\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.383073 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.383036    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-trusted-ca-bundle\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.383250 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.383097    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/824f84de-51ea-4435-9aee-9fbcde961d41-console-serving-cert\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.383250 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.383123    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-service-ca\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.383250 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.383153    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/824f84de-51ea-4435-9aee-9fbcde961d41-console-oauth-config\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.383250 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.383176    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-oauth-serving-cert\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.383888 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.383867    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-oauth-serving-cert\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.384702 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.384684    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-console-config\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.385284 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.385142    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/824f84de-51ea-4435-9aee-9fbcde961d41-tmp\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.385741 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.385657    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-service-ca\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.385741 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.385723    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-trusted-ca-bundle\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.388040 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.387996    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/824f84de-51ea-4435-9aee-9fbcde961d41-console-serving-cert\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.388744 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.388703    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/824f84de-51ea-4435-9aee-9fbcde961d41-console-oauth-config\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.447083 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.447018    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4z99\" (UniqueName: \"kubernetes.io/projected/824f84de-51ea-4435-9aee-9fbcde961d41-kube-api-access-t4z99\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:34.453214 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.453113    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7445978df6-sf44q" event={"ID":"cf17e11d-1428-47c8-9aea-9f6b7c831eb1","Type":"ContainerStarted","Data":"48aec6da0f776fb642caeda9775dc3cb704583591bc8a7d7fe453b8c84663736"}
Oct 30 18:34:34.457930 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.456892    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-h5nm6" event={"ID":"08a8e709-63af-4d71-b084-cf533ea620e5","Type":"ContainerStarted","Data":"a8e9b9ab990864f004eb90fa1b369c35e461ecd4db418f7b9e7471eef9a7b0c8"}
Oct 30 18:34:34.457930 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.456939    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-h5nm6" event={"ID":"08a8e709-63af-4d71-b084-cf533ea620e5","Type":"ContainerStarted","Data":"b9537390b504df482f3713ff8eda0baf44a0a197cfdfce5b48651f3662596c1f"}
Oct 30 18:34:34.458995 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.458155    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:34.458995 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.458780    2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:34.462746 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.462699    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j" event={"ID":"dc7462f0-a45f-487a-99b9-3e89f783cb01","Type":"ContainerStarted","Data":"e24685c45cd1c1f042e3505248a05f1579704e97d3d9750ca6a6bcd82611622a"}
Oct 30 18:34:34.465380 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.465333    2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:34.559346 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.559274    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:34.697464 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.697362    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-h5nm6" podStartSLOduration=66.94769367 podStartE2EDuration="1m8.697342179s" totalImagesPullingTime="1.749648509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:34:34.576043183 +0000 UTC m=+69.012307313" watchObservedRunningTime="2026-10-30 18:34:34.697342179 +0000 UTC m=+69.133606313"
Oct 30 18:34:34.741248 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:34.740912    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:35.142331 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:35.142301    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59c4fb9dcd-xzxdr"]
Oct 30 18:34:35.147291 ip-10-0-133-80 kubenswrapper[2577]: W1030 18:34:35.147263    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod824f84de_51ea_4435_9aee_9fbcde961d41.slice/crio-0515afe254d6c0481584c8483f6a489eca877016a7560058b6a645a864c7534a WatchSource:0}: Error finding container 0515afe254d6c0481584c8483f6a489eca877016a7560058b6a645a864c7534a: Status 404 returned error can't find the container with id 0515afe254d6c0481584c8483f6a489eca877016a7560058b6a645a864c7534a
Oct 30 18:34:35.474983 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:35.474799    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c4fb9dcd-xzxdr" event={"ID":"824f84de-51ea-4435-9aee-9fbcde961d41","Type":"ContainerStarted","Data":"d84ca3f135c80df8ac7a13e40589d5cd5b485bf1f3846846350988f1089020f4"}
Oct 30 18:34:35.474983 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:35.474874    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c4fb9dcd-xzxdr" event={"ID":"824f84de-51ea-4435-9aee-9fbcde961d41","Type":"ContainerStarted","Data":"0515afe254d6c0481584c8483f6a489eca877016a7560058b6a645a864c7534a"}
Oct 30 18:34:35.486454 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:35.481985    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd" event={"ID":"220393aa-93ee-4113-b373-ecd27198f170","Type":"ContainerStarted","Data":"0511e6c91efd4ab6871e904c809cdd95b990e99e956c869e31b80278e6571f14"}
Oct 30 18:34:35.486454 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:35.482056    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd" event={"ID":"220393aa-93ee-4113-b373-ecd27198f170","Type":"ContainerStarted","Data":"6f04f6ae9fbbff061a2770d47eae947288503ac8f8591f44364db7f9c791bae9"}
Oct 30 18:34:35.488848 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:35.488795    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:35.722744 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:35.722442    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-59c4fb9dcd-xzxdr" podStartSLOduration=1.7224214660000001 podStartE2EDuration="1.722421466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:34:35.597102565 +0000 UTC m=+70.033366696" watchObservedRunningTime="2026-10-30 18:34:35.722421466 +0000 UTC m=+70.158685598"
Oct 30 18:34:36.498773 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:36.498743    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd" event={"ID":"220393aa-93ee-4113-b373-ecd27198f170","Type":"ContainerStarted","Data":"3ec5b4b3e273cf66c54bb3f947335736801af31554cc90e6b8520a3d06c87297"}
Oct 30 18:34:36.511994 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:36.511538    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7ed38b7-0997-4cd8-9da0-41093596a6a0","Type":"ContainerStarted","Data":"43a340d025c45535e367a6aea3e41cdb52b3bde8dd27e6f372180e2e436f5680"}
Oct 30 18:34:36.511994 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:36.511613    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7ed38b7-0997-4cd8-9da0-41093596a6a0","Type":"ContainerStarted","Data":"812c6cf749ef9fd286430ba652f60a717c618ee8d85654b467b446cd4fb6b8db"}
Oct 30 18:34:36.511994 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:36.511658    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7ed38b7-0997-4cd8-9da0-41093596a6a0","Type":"ContainerStarted","Data":"7fdca55ec4308700260efd57276b0518d1eb3d50e1eada4def6468a83c846095"}
Oct 30 18:34:37.518280 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:37.518166    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5gvjh" event={"ID":"a864943c-d7eb-4486-99a6-7961573b57d4","Type":"ContainerStarted","Data":"25d149af8910e47926eee39b907015d8d08d669129df1769106f6d8081aba904"}
Oct 30 18:34:37.518280 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:37.518251    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:34:37.521602 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:37.521200    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j" event={"ID":"dc7462f0-a45f-487a-99b9-3e89f783cb01","Type":"ContainerStarted","Data":"dd8ca644274d872b9e5b407ecffcfc9400f5fd22446cd1104fbc423c38189113"}
Oct 30 18:34:37.521602 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:37.521502    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j"
Oct 30 18:34:37.526354 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:37.526181    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd" event={"ID":"220393aa-93ee-4113-b373-ecd27198f170","Type":"ContainerStarted","Data":"797999925ed5804e7c8c5d16c45dcca51d7a8a74d9dfa915fd9d11681de4561e"}
Oct 30 18:34:37.526354 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:37.526248    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd" event={"ID":"220393aa-93ee-4113-b373-ecd27198f170","Type":"ContainerStarted","Data":"5f952a18615ab9dee80cf60de393f91dc10dfd7cc7d2fb31b122d773b4bbaa1d"}
Oct 30 18:34:37.526354 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:37.526290    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd" event={"ID":"220393aa-93ee-4113-b373-ecd27198f170","Type":"ContainerStarted","Data":"5b58c54d6e53d7ac5bd28dc0eb8445f5b2210a6855abc4aa312dc6a901fd19ab"}
Oct 30 18:34:37.526354 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:37.526344    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:37.528685 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:37.528667    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j"
Oct 30 18:34:37.530764 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:37.530647    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7ed38b7-0997-4cd8-9da0-41093596a6a0","Type":"ContainerStarted","Data":"f7a1ef9cc808c0dc378f29e3ac1afd1b1833574b00dda97ae81661cc76c966c1"}
Oct 30 18:34:37.530764 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:37.530713    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7ed38b7-0997-4cd8-9da0-41093596a6a0","Type":"ContainerStarted","Data":"ad5dc45c6ed151741af41b6243ad6704239b5ba43a4a62af9cff03c6b57983bd"}
Oct 30 18:34:37.532937 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:37.532416    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7445978df6-sf44q" event={"ID":"cf17e11d-1428-47c8-9aea-9f6b7c831eb1","Type":"ContainerStarted","Data":"87713a208ab9f1d31a757dbdbe1c98c3dc22d0bef85278761dd838ab970ef24a"}
Oct 30 18:34:37.536198 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:37.536154    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:37.636291 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:37.636196    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5gvjh" podStartSLOduration=66.320041665 podStartE2EDuration="1m11.636183718s" totalImagesPullingTime="5.316142053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:34:37.635362797 +0000 UTC m=+72.071626928" watchObservedRunningTime="2026-10-30 18:34:37.636183718 +0000 UTC m=+72.072447848"
Oct 30 18:34:37.779173 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:37.779122    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd" podStartSLOduration=1.806222607 podStartE2EDuration="6.779101686s" totalImagesPullingTime="4.972879079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:34:37.778358497 +0000 UTC m=+72.214622628" watchObservedRunningTime="2026-10-30 18:34:37.779101686 +0000 UTC m=+72.215365817"
Oct 30 18:34:37.896894 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:37.896795    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j" podStartSLOduration=1.9804238299999999 podStartE2EDuration="4.89677602s" totalImagesPullingTime="2.91635219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:34:37.896081273 +0000 UTC m=+72.332345405" watchObservedRunningTime="2026-10-30 18:34:37.89677602 +0000 UTC m=+72.333040152"
Oct 30 18:34:38.018293 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:38.018230    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7445978df6-sf44q" podStartSLOduration=1.908759765 podStartE2EDuration="5.01821547s" totalImagesPullingTime="3.109455705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:34:38.017782866 +0000 UTC m=+72.454046997" watchObservedRunningTime="2026-10-30 18:34:38.01821547 +0000 UTC m=+72.454479599"
Oct 30 18:34:38.546926 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:38.546895    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7ed38b7-0997-4cd8-9da0-41093596a6a0","Type":"ContainerStarted","Data":"ecb8816873ee3bc29892301cdf67a02f36a68e05b30f5ce97a93854c9c8e4812"}
Oct 30 18:34:38.678530 ip-10-0-133-80 kubenswrapper[2577]: I1030 18:34:38.678463    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.55478227 podStartE2EDuration="8.678443162s" totalImagesPullingTime="5.104389385s" totalInitContainerRuntime="19.271507ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:34:38.67690783 +0000 UTC m=+73.113171961" watchObservedRunningTime="2026-10-30 18:34:38.678443162 +0000 UTC m=+73.114707292"
Oct 30 18:34:39.204153 ip-10-0-133-80 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 18:34:39.220444 ip-10-0-133-80 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 18:34:39.220614 ip-10-0-133-80 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 18:34:39.220666 ip-10-0-133-80 systemd[1]: kubelet.service: Consumed 4.463s CPU time.
Oct 30 18:34:39.236790 ip-10-0-133-80 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 18:34:39.316621 ip-10-0-133-80 kubenswrapper[5820]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:34:39.316621 ip-10-0-133-80 kubenswrapper[5820]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 18:34:39.316621 ip-10-0-133-80 kubenswrapper[5820]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:34:39.316621 ip-10-0-133-80 kubenswrapper[5820]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:34:39.319883 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.319791    5820 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:34:39.319883 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.319886    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:34:39.319971 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.319891    5820 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:34:39.319971 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.319897    5820 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:34:39.319971 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.319901    5820 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:34:39.319971 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.319906    5820 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:34:39.319971 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.319910    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:34:39.319971 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.319915    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:34:39.319971 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.319920    5820 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:34:39.319971 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.319924    5820 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:34:39.319971 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.319928    5820 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:34:39.319971 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.319932    5820 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:34:39.319971 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.319936    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:34:39.319971 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.319940    5820 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:34:39.319971 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.319966    5820 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:34:39.319971 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.319976    5820 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:34:39.320387 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.319980    5820 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:34:39.320387 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.319985    5820 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:34:39.320387 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.319988    5820 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:34:39.320387 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.319993    5820 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:34:39.320387 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.319997    5820 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:34:39.320387 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320001    5820 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:34:39.320387 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320005    5820 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:34:39.320387 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320009    5820 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:34:39.320387 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320013    5820 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:34:39.320387 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320017    5820 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:34:39.320387 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320021    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:34:39.320387 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320025    5820 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:34:39.320387 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320029    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:34:39.320387 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320033    5820 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:34:39.320387 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320037    5820 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:34:39.320387 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320040    5820 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:34:39.320387 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320044    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:34:39.320387 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320048    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:34:39.320387 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320054    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:34:39.320870 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320058    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:34:39.320870 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320062    5820 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:34:39.320870 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320066    5820 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:34:39.320870 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320070    5820 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:34:39.320870 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320074    5820 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:34:39.320870 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320078    5820 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:34:39.320870 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320082    5820 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:34:39.320870 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320086    5820 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:34:39.320870 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320090    5820 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:34:39.320870 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320093    5820 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:34:39.320870 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320098    5820 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:34:39.320870 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320104    5820 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:34:39.320870 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320108    5820 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:34:39.320870 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320112    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:34:39.320870 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320117    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:34:39.320870 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320121    5820 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:34:39.320870 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320127    5820 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:34:39.320870 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320133    5820 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:34:39.320870 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320139    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:34:39.321605 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320144    5820 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:34:39.321605 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320150    5820 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:34:39.321605 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320154    5820 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:34:39.321605 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320158    5820 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:34:39.321605 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320163    5820 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:34:39.321605 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320168    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:34:39.321605 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320174    5820 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:34:39.321605 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320180    5820 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:34:39.321605 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320184    5820 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:34:39.321605 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320189    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:34:39.321605 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320194    5820 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:34:39.321605 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320199    5820 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:34:39.321605 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320203    5820 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:34:39.321605 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320207    5820 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:34:39.321605 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320212    5820 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:34:39.321605 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320216    5820 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:34:39.321605 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320221    5820 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:34:39.321605 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320225    5820 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:34:39.321605 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320229    5820 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:34:39.322362 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320233    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:34:39.322362 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320237    5820 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:34:39.322362 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320241    5820 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:34:39.322362 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320246    5820 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:34:39.322362 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320250    5820 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:34:39.322362 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320254    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:34:39.322362 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320260    5820 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:34:39.322362 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320265    5820 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:34:39.322362 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320272    5820 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:34:39.322362 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320275    5820 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:34:39.322362 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320279    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:34:39.322362 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320283    5820 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:34:39.322362 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320287    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:34:39.322362 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320291    5820 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:34:39.322362 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320296    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:34:39.322362 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320300    5820 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:34:39.322362 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320305    5820 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:34:39.322362 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320310    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:34:39.322362 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320314    5820 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:34:39.323105 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320318    5820 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:34:39.323105 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320322    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:34:39.323105 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320328    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:34:39.323105 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320332    5820 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:34:39.323105 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320337    5820 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:34:39.323105 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320341    5820 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:34:39.323105 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320344    5820 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:34:39.323105 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320349    5820 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:34:39.323105 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320353    5820 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:34:39.323105 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320357    5820 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:34:39.323105 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320360    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:34:39.323105 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320365    5820 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:34:39.323105 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320376    5820 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:34:39.323105 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.320381    5820 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:34:39.323105 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321550    5820 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:34:39.323105 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321560    5820 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:34:39.323105 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321564    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:34:39.323105 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321569    5820 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:34:39.323105 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321573    5820 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:34:39.323105 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321577    5820 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:34:39.323770 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321582    5820 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:34:39.323770 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321587    5820 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:34:39.323770 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321591    5820 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:34:39.323770 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321596    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:34:39.323770 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321600    5820 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:34:39.323770 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321604    5820 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:34:39.323770 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321608    5820 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:34:39.323770 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321612    5820 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:34:39.323770 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321617    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:34:39.323770 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321621    5820 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:34:39.323770 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321625    5820 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:34:39.323770 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321629    5820 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:34:39.323770 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321633    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:34:39.323770 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321637    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:34:39.323770 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321641    5820 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:34:39.323770 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321644    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:34:39.323770 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321648    5820 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:34:39.323770 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321652    5820 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:34:39.323770 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321656    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:34:39.324394 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321659    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:34:39.324394 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321663    5820 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:34:39.324394 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321667    5820 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:34:39.324394 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321671    5820 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:34:39.324394 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321675    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:34:39.324394 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321679    5820 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:34:39.324394 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321683    5820 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:34:39.324394 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321687    5820 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:34:39.324394 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321690    5820 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:34:39.324394 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321694    5820 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:34:39.324394 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321698    5820 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:34:39.324394 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321702    5820 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:34:39.324394 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321706    5820 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:34:39.324394 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321710    5820 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:34:39.324394 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321721    5820 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:34:39.324394 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321725    5820 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:34:39.324394 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321729    5820 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:34:39.324394 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321733    5820 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:34:39.324394 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321737    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:34:39.324394 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321740    5820 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:34:39.324935 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321744    5820 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:34:39.324935 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321748    5820 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:34:39.324935 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321752    5820 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:34:39.324935 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321756    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:34:39.324935 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321760    5820 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:34:39.324935 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321764    5820 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:34:39.324935 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321768    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:34:39.324935 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321772    5820 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:34:39.324935 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321776    5820 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:34:39.324935 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321781    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:34:39.324935 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321785    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:34:39.324935 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321789    5820 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:34:39.324935 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321796    5820 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:34:39.324935 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321802    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:34:39.324935 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321808    5820 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:34:39.324935 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321812    5820 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:34:39.324935 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321821    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:34:39.324935 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321824    5820 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:34:39.324935 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321828    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:34:39.325421 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321832    5820 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:34:39.325421 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321836    5820 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:34:39.325421 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321840    5820 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:34:39.325421 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321843    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:34:39.325421 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321847    5820 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:34:39.325421 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321851    5820 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:34:39.325421 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321855    5820 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:34:39.325421 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321859    5820 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:34:39.325421 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321870    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:34:39.325421 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321874    5820 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:34:39.325421 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321877    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:34:39.325421 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321882    5820 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:34:39.325421 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321885    5820 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:34:39.325421 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321889    5820 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:34:39.325421 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321893    5820 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:34:39.325421 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321896    5820 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:34:39.325421 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321900    5820 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:34:39.325421 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321904    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:34:39.325421 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321908    5820 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:34:39.325926 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321912    5820 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:34:39.325926 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321916    5820 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:34:39.325926 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321920    5820 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:34:39.325926 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321923    5820 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:34:39.325926 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321928    5820 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:34:39.325926 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321932    5820 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:34:39.325926 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321936    5820 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:34:39.325926 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321939    5820 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:34:39.325926 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321943    5820 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:34:39.325926 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321968    5820 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:34:39.325926 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321974    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:34:39.325926 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321978    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:34:39.325926 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321981    5820 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:34:39.325926 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321985    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:34:39.325926 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321989    5820 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:34:39.325926 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321993    5820 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:34:39.325926 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.321997    5820 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:34:39.325926 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322001    5820 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:34:39.325926 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322006    5820 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:34:39.325926 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322009    5820 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:34:39.326438 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322013    5820 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:34:39.326438 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322019    5820 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:34:39.326438 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322034    5820 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:34:39.326438 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322784    5820 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:34:39.326438 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322793    5820 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:34:39.326438 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322798    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:34:39.326438 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322803    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:34:39.326438 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322807    5820 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:34:39.326438 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322811    5820 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:34:39.326438 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322815    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:34:39.326438 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322820    5820 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:34:39.326438 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322824    5820 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:34:39.326438 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322828    5820 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:34:39.326438 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322833    5820 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:34:39.326438 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322838    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:34:39.326438 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322842    5820 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:34:39.326438 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322846    5820 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:34:39.326438 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322850    5820 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:34:39.326438 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322854    5820 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:34:39.326900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322858    5820 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:34:39.326900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322862    5820 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:34:39.326900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322866    5820 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:34:39.326900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322872    5820 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:34:39.326900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322876    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:34:39.326900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322880    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:34:39.326900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322884    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:34:39.326900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322888    5820 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:34:39.326900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322892    5820 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:34:39.326900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322899    5820 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:34:39.326900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322905    5820 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:34:39.326900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322909    5820 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:34:39.326900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322914    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:34:39.326900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322918    5820 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:34:39.326900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322922    5820 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:34:39.326900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322927    5820 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:34:39.326900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322931    5820 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:34:39.326900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322935    5820 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:34:39.326900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322939    5820 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:34:39.326900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322962    5820 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:34:39.327424 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322968    5820 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:34:39.327424 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322972    5820 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:34:39.327424 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322976    5820 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:34:39.327424 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322980    5820 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:34:39.327424 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322984    5820 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:34:39.327424 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322988    5820 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:34:39.327424 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322992    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:34:39.327424 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.322997    5820 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:34:39.327424 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323001    5820 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:34:39.327424 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323005    5820 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:34:39.327424 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323009    5820 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:34:39.327424 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323013    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:34:39.327424 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323016    5820 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:34:39.327424 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323020    5820 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:34:39.327424 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323024    5820 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:34:39.327424 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323029    5820 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:34:39.327424 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323034    5820 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:34:39.327424 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323038    5820 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:34:39.327424 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323041    5820 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:34:39.327424 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323045    5820 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:34:39.327900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323049    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:34:39.327900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323054    5820 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:34:39.327900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323058    5820 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:34:39.327900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323062    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:34:39.327900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323066    5820 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:34:39.327900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323070    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:34:39.327900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323074    5820 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:34:39.327900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323078    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:34:39.327900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323083    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:34:39.327900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323087    5820 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:34:39.327900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323091    5820 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:34:39.327900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323095    5820 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:34:39.327900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323099    5820 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:34:39.327900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323103    5820 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:34:39.327900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323107    5820 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:34:39.327900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323111    5820 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:34:39.327900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323115    5820 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:34:39.327900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323119    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:34:39.327900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323124    5820 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:34:39.327900 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323127    5820 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:34:39.328674 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323131    5820 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:34:39.328674 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323135    5820 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:34:39.328674 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323139    5820 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:34:39.328674 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323142    5820 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:34:39.328674 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323146    5820 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:34:39.328674 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323150    5820 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:34:39.328674 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323154    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:34:39.328674 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323159    5820 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:34:39.328674 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323163    5820 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:34:39.328674 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323167    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:34:39.328674 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323171    5820 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:34:39.328674 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323175    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:34:39.328674 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323179    5820 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:34:39.328674 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323184    5820 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:34:39.328674 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323188    5820 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:34:39.328674 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323192    5820 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:34:39.328674 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323195    5820 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:34:39.328674 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323199    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:34:39.328674 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323204    5820 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:34:39.329430 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323208    5820 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:34:39.329430 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323212    5820 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:34:39.329430 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323216    5820 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:34:39.329430 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323220    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:34:39.329430 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323224    5820 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:34:39.329430 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323228    5820 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:34:39.329430 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323232    5820 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:34:39.329430 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323236    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:34:39.329430 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323239    5820 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:34:39.329430 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323243    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:34:39.329430 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.323247    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:34:39.329430 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323380    5820 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 18:34:39.329430 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323391    5820 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 18:34:39.329430 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323404    5820 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 18:34:39.329430 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323409    5820 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 18:34:39.329430 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323413    5820 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 18:34:39.329430 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323417    5820 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 18:34:39.329430 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323423    5820 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 18:34:39.329430 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323428    5820 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 18:34:39.329430 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323434    5820 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 18:34:39.329430 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323438    5820 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323441    5820 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323444    5820 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323447    5820 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323450    5820 flags.go:64] FLAG: --cgroup-root=""
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323453    5820 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323457    5820 flags.go:64] FLAG: --client-ca-file=""
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323460    5820 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323463    5820 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323469    5820 flags.go:64] FLAG: --cluster-domain=""
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323472    5820 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323477    5820 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323482    5820 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323488    5820 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323493    5820 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323504    5820 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323510    5820 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323515    5820 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323518    5820 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323521    5820 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323524    5820 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323527    5820 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323530    5820 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323534    5820 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 18:34:39.330278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323537    5820 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323540    5820 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323543    5820 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323545    5820 flags.go:64] FLAG: --enable-server="true"
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323548    5820 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323582    5820 flags.go:64] FLAG: --event-burst="100"
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323588    5820 flags.go:64] FLAG: --event-qps="50"
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323594    5820 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323597    5820 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323600    5820 flags.go:64] FLAG: --eviction-hard=""
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323605    5820 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323608    5820 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323611    5820 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323613    5820 flags.go:64] FLAG: --eviction-soft=""
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323618    5820 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323621    5820 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323624    5820 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323626    5820 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323629    5820 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323632    5820 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323635    5820 flags.go:64] FLAG: --feature-gates=""
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323638    5820 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323641    5820 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323644    5820 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323653    5820 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323657    5820 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 18:34:39.330931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323662    5820 flags.go:64] FLAG: --help="false"
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323667    5820 flags.go:64] FLAG: --hostname-override="ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323672    5820 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323676    5820 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323679    5820 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323682    5820 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323685    5820 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323688    5820 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323690    5820 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323693    5820 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323696    5820 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323698    5820 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323704    5820 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323707    5820 flags.go:64] FLAG: --kube-reserved=""
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323710    5820 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323713    5820 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323716    5820 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323718    5820 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323721    5820 flags.go:64] FLAG: --lock-file=""
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323724    5820 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323728    5820 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323731    5820 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323737    5820 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 18:34:39.331924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323742    5820 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323747    5820 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323752    5820 flags.go:64] FLAG: --logging-format="text"
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323755    5820 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323758    5820 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323761    5820 flags.go:64] FLAG: --manifest-url=""
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323764    5820 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323769    5820 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323772    5820 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323777    5820 flags.go:64] FLAG: --max-pods="110"
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323780    5820 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323783    5820 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323786    5820 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323789    5820 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323792    5820 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323795    5820 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323798    5820 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323805    5820 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323808    5820 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323811    5820 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323815    5820 flags.go:64] FLAG: --pod-cidr=""
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323823    5820 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323827    5820 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323832    5820 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323835    5820 flags.go:64] FLAG: --port="10250"
Oct 30 18:34:39.332523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323838    5820 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323840    5820 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-09e0ef491dbe1b3b7"
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323843    5820 flags.go:64] FLAG: --qos-reserved=""
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323846    5820 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323854    5820 flags.go:64] FLAG: --register-node="true"
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323856    5820 flags.go:64] FLAG: --register-with-taints=""
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323860    5820 flags.go:64] FLAG: --registry-burst="10"
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323863    5820 flags.go:64] FLAG: --registry-qps="5"
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323866    5820 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323869    5820 flags.go:64] FLAG: --reserved-memory=""
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323873    5820 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323876    5820 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323879    5820 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323881    5820 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323888    5820 flags.go:64] FLAG: --runonce="false"
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323891    5820 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323905    5820 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323909    5820 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323915    5820 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323919    5820 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323921    5820 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323924    5820 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323927    5820 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323930    5820 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323933    5820 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 18:34:39.333131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323936    5820 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323939    5820 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323942    5820 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323967    5820 flags.go:64] FLAG: --system-cgroups=""
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323973    5820 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323981    5820 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323983    5820 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323986    5820 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323993    5820 flags.go:64] FLAG: --tls-min-version=""
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323996    5820 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.323999    5820 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.324004    5820 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.324007    5820 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.324010    5820 flags.go:64] FLAG: --v="2"
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.324014    5820 flags.go:64] FLAG: --version="false"
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.324018    5820 flags.go:64] FLAG: --vmodule=""
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.324022    5820 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.324025    5820 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324212    5820 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324219    5820 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324224    5820 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324228    5820 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324234    5820 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324238    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:34:39.333719 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324242    5820 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:34:39.334324 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324245    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:34:39.334324 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324248    5820 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:34:39.334324 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324250    5820 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:34:39.334324 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324255    5820 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:34:39.334324 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324258    5820 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:34:39.334324 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324261    5820 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:34:39.334324 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324264    5820 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:34:39.334324 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324266    5820 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:34:39.334324 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324269    5820 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:34:39.334324 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324271    5820 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:34:39.334324 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324276    5820 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:34:39.334324 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324278    5820 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:34:39.334324 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324281    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:34:39.334324 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324283    5820 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:34:39.334324 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324286    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:34:39.334324 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324289    5820 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:34:39.334324 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324291    5820 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:34:39.334324 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324295    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:34:39.334324 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324302    5820 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:34:39.334778 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324306    5820 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:34:39.334778 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324311    5820 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:34:39.334778 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324315    5820 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:34:39.334778 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324319    5820 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:34:39.334778 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324322    5820 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:34:39.334778 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324325    5820 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:34:39.334778 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324328    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:34:39.334778 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324330    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:34:39.334778 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324333    5820 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:34:39.334778 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324336    5820 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:34:39.334778 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324338    5820 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:34:39.334778 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324340    5820 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:34:39.334778 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324343    5820 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:34:39.334778 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324351    5820 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:34:39.334778 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324354    5820 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:34:39.334778 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324357    5820 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:34:39.334778 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324359    5820 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:34:39.334778 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324362    5820 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:34:39.334778 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324364    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:34:39.334778 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324377    5820 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:34:39.335311 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324381    5820 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:34:39.335311 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324386    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:34:39.335311 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324390    5820 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:34:39.335311 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324397    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:34:39.335311 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324402    5820 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:34:39.335311 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324406    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:34:39.335311 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324408    5820 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:34:39.335311 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324411    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:34:39.335311 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324414    5820 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:34:39.335311 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324416    5820 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:34:39.335311 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324419    5820 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:34:39.335311 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324422    5820 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:34:39.335311 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324425    5820 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:34:39.335311 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324427    5820 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:34:39.335311 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324430    5820 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:34:39.335311 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324432    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:34:39.335311 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324435    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:34:39.335311 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324437    5820 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:34:39.335311 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324440    5820 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:34:39.335311 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324443    5820 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:34:39.336143 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324445    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:34:39.336143 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324448    5820 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:34:39.336143 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324450    5820 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:34:39.336143 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324453    5820 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:34:39.336143 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324455    5820 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:34:39.336143 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324458    5820 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:34:39.336143 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324468    5820 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:34:39.336143 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324473    5820 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:34:39.336143 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324477    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:34:39.336143 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324481    5820 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:34:39.336143 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324485    5820 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:34:39.336143 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324488    5820 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:34:39.336143 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324491    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:34:39.336143 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324493    5820 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:34:39.336143 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324496    5820 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:34:39.336143 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324502    5820 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:34:39.336143 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324506    5820 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:34:39.336143 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324509    5820 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:34:39.336143 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324512    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:34:39.336143 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324514    5820 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:34:39.336890 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324517    5820 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:34:39.336890 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324519    5820 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:34:39.336890 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324521    5820 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:34:39.336890 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324524    5820 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:34:39.336890 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324527    5820 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:34:39.336890 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324529    5820 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:34:39.336890 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324532    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:34:39.336890 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324534    5820 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:34:39.336890 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324537    5820 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:34:39.336890 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324539    5820 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:34:39.336890 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324542    5820 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:34:39.336890 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324546    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:34:39.336890 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324550    5820 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:34:39.336890 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324554    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:34:39.336890 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324559    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:34:39.336890 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324563    5820 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:34:39.336890 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324566    5820 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:34:39.336890 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324570    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:34:39.336890 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324573    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:34:39.337639 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324581    5820 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:34:39.337639 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.324618    5820 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:34:39.337639 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324673    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:34:39.337639 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324678    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:34:39.337639 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324681    5820 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:34:39.337639 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324684    5820 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:34:39.337639 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324686    5820 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:34:39.337639 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324689    5820 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:34:39.337639 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324691    5820 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:34:39.337639 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324695    5820 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:34:39.337639 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324698    5820 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:34:39.337639 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324701    5820 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:34:39.337639 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324703    5820 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:34:39.337639 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324706    5820 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:34:39.337639 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324708    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:34:39.337639 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324711    5820 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:34:39.338126 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324715    5820 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:34:39.338126 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324720    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:34:39.338126 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324724    5820 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:34:39.338126 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324729    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:34:39.338126 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324733    5820 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:34:39.338126 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324737    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:34:39.338126 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324740    5820 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:34:39.338126 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324742    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:34:39.338126 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324745    5820 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:34:39.338126 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324748    5820 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:34:39.338126 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324750    5820 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:34:39.338126 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324753    5820 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:34:39.338126 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324755    5820 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:34:39.338126 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324758    5820 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:34:39.338126 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324760    5820 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:34:39.338126 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324763    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:34:39.338126 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324765    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:34:39.338126 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324773    5820 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:34:39.338126 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324776    5820 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:34:39.338126 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324779    5820 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:34:39.338682 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324781    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:34:39.338682 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324784    5820 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:34:39.338682 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324786    5820 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:34:39.338682 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324789    5820 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:34:39.338682 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324791    5820 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:34:39.338682 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324796    5820 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:34:39.338682 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324801    5820 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:34:39.338682 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324805    5820 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:34:39.338682 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324810    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:34:39.338682 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324814    5820 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:34:39.338682 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324817    5820 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:34:39.338682 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324821    5820 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:34:39.338682 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324824    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:34:39.338682 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324827    5820 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:34:39.338682 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324830    5820 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:34:39.338682 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324834    5820 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:34:39.338682 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324838    5820 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:34:39.338682 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324841    5820 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:34:39.338682 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324844    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:34:39.338682 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324846    5820 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:34:39.339465 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324849    5820 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:34:39.339465 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324851    5820 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:34:39.339465 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324854    5820 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:34:39.339465 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324856    5820 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:34:39.339465 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324859    5820 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:34:39.339465 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324861    5820 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:34:39.339465 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324864    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:34:39.339465 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324866    5820 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:34:39.339465 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324869    5820 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:34:39.339465 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324871    5820 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:34:39.339465 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324882    5820 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:34:39.339465 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324886    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:34:39.339465 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324890    5820 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:34:39.339465 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324895    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:34:39.339465 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324899    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:34:39.339465 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324904    5820 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:34:39.339465 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324907    5820 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:34:39.339465 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324910    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:34:39.339465 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324913    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:34:39.340008 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324915    5820 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:34:39.340008 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324918    5820 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:34:39.340008 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324920    5820 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:34:39.340008 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324923    5820 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:34:39.340008 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324925    5820 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:34:39.340008 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324930    5820 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:34:39.340008 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324932    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:34:39.340008 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324935    5820 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:34:39.340008 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324938    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:34:39.340008 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324941    5820 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:34:39.340008 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324958    5820 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:34:39.340008 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324963    5820 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:34:39.340008 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324970    5820 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:34:39.340008 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324977    5820 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:34:39.340008 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324982    5820 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:34:39.340008 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324986    5820 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:34:39.340008 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324988    5820 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:34:39.340008 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324991    5820 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:34:39.340008 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324994    5820 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:34:39.340485 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324996    5820 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:34:39.340485 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.324999    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:34:39.340485 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.325001    5820 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:34:39.340485 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.325004    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:34:39.340485 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.325006    5820 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:34:39.340485 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.325015    5820 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:34:39.340485 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.325018    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:34:39.340485 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.325021    5820 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:34:39.340485 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.325023    5820 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:34:39.340485 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.325025    5820 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:34:39.340485 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.325029    5820 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:34:39.340485 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.325033    5820 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:34:39.340485 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.325036    5820 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:34:39.340485 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.325038    5820 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:34:39.340485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.332247    5820 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 18:34:39.340485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.332264    5820 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 18:34:39.340485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.332291    5820 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 18:34:39.340485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.332296    5820 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 18:34:39.340485 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332357    5820 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:34:39.340941 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332362    5820 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:34:39.340941 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332374    5820 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:34:39.340941 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332378    5820 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:34:39.340941 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332381    5820 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:34:39.340941 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332383    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:34:39.340941 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332386    5820 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:34:39.340941 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332389    5820 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:34:39.340941 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332392    5820 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:34:39.340941 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332395    5820 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:34:39.340941 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332399    5820 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:34:39.340941 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332402    5820 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:34:39.340941 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332405    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:34:39.340941 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332408    5820 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:34:39.340941 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332411    5820 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:34:39.340941 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332413    5820 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:34:39.340941 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332416    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:34:39.340941 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332419    5820 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:34:39.340941 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332421    5820 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:34:39.340941 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332424    5820 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:34:39.341420 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332426    5820 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:34:39.341420 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332429    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:34:39.341420 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332431    5820 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:34:39.341420 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332434    5820 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:34:39.341420 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332436    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:34:39.341420 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332439    5820 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:34:39.341420 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332442    5820 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:34:39.341420 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332444    5820 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:34:39.341420 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332448    5820 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:34:39.341420 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332451    5820 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:34:39.341420 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332454    5820 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:34:39.341420 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332457    5820 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:34:39.341420 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332459    5820 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:34:39.341420 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332462    5820 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:34:39.341420 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332464    5820 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:34:39.341420 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332467    5820 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:34:39.341420 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332469    5820 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:34:39.341420 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332472    5820 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:34:39.341420 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332475    5820 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:34:39.341420 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332478    5820 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:34:39.342032 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332480    5820 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:34:39.342032 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332483    5820 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:34:39.342032 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332486    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:34:39.342032 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332488    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:34:39.342032 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332491    5820 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:34:39.342032 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332493    5820 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:34:39.342032 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332496    5820 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:34:39.342032 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332499    5820 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:34:39.342032 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332501    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:34:39.342032 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332504    5820 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:34:39.342032 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332506    5820 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:34:39.342032 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332509    5820 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:34:39.342032 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332511    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:34:39.342032 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332514    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:34:39.342032 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332516    5820 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:34:39.342032 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332519    5820 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:34:39.342032 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332521    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:34:39.342032 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332524    5820 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:34:39.342032 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332527    5820 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:34:39.342032 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332529    5820 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:34:39.342669 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332532    5820 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:34:39.342669 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332534    5820 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:34:39.342669 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332537    5820 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:34:39.342669 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332540    5820 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:34:39.342669 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332543    5820 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:34:39.342669 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332545    5820 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:34:39.342669 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332548    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:34:39.342669 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332550    5820 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:34:39.342669 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332553    5820 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:34:39.342669 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332556    5820 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:34:39.342669 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332558    5820 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:34:39.342669 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332561    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:34:39.342669 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332563    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:34:39.342669 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332566    5820 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:34:39.342669 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332569    5820 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:34:39.342669 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332571    5820 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:34:39.342669 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332574    5820 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:34:39.342669 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332576    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:34:39.342669 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332579    5820 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:34:39.343261 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332582    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:34:39.343261 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332585    5820 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:34:39.343261 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332587    5820 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:34:39.343261 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332590    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:34:39.343261 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332592    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:34:39.343261 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332595    5820 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:34:39.343261 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332597    5820 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:34:39.343261 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332600    5820 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:34:39.343261 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332602    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:34:39.343261 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332605    5820 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:34:39.343261 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332607    5820 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:34:39.343261 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332610    5820 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:34:39.343261 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332612    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:34:39.343261 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332616    5820 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:34:39.343261 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332620    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:34:39.343261 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332623    5820 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:34:39.343261 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332626    5820 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:34:39.343261 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332630    5820 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:34:39.343261 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332632    5820 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:34:39.343261 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332635    5820 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:34:39.343786 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332638    5820 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:34:39.343786 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332640    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:34:39.343786 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332643    5820 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:34:39.343786 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332646    5820 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:34:39.343786 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332648    5820 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:34:39.343786 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332651    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:34:39.343786 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332654    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:34:39.343786 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.332683    5820 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:34:39.343786 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332818    5820 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:34:39.343786 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332822    5820 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:34:39.343786 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332825    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:34:39.343786 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332827    5820 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:34:39.343786 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332831    5820 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:34:39.343786 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332834    5820 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:34:39.343786 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332837    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:34:39.344183 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332839    5820 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:34:39.344183 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332842    5820 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:34:39.344183 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332844    5820 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:34:39.344183 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332847    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:34:39.344183 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332849    5820 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:34:39.344183 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332852    5820 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:34:39.344183 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332854    5820 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:34:39.344183 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332857    5820 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:34:39.344183 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332859    5820 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:34:39.344183 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332862    5820 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:34:39.344183 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332864    5820 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:34:39.344183 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332867    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:34:39.344183 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332870    5820 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:34:39.344183 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332873    5820 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:34:39.344183 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332875    5820 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:34:39.344183 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332878    5820 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:34:39.344183 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332880    5820 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:34:39.344183 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332882    5820 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:34:39.344183 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332885    5820 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:34:39.344183 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332887    5820 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:34:39.344722 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332890    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:34:39.344722 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332893    5820 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:34:39.344722 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332897    5820 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:34:39.344722 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332900    5820 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:34:39.344722 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332902    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:34:39.344722 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332905    5820 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:34:39.344722 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332907    5820 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:34:39.344722 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332910    5820 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:34:39.344722 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332912    5820 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:34:39.344722 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332914    5820 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:34:39.344722 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332917    5820 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:34:39.344722 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332920    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:34:39.344722 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332922    5820 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:34:39.344722 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332925    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:34:39.344722 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332927    5820 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:34:39.344722 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332929    5820 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:34:39.344722 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332932    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:34:39.344722 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332934    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:34:39.344722 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332937    5820 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:34:39.345474 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332940    5820 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:34:39.345474 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332942    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:34:39.345474 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332964    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:34:39.345474 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332966    5820 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:34:39.345474 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332969    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:34:39.345474 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332972    5820 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:34:39.345474 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332975    5820 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:34:39.345474 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332977    5820 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:34:39.345474 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332980    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:34:39.345474 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332982    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:34:39.345474 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332985    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:34:39.345474 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332987    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:34:39.345474 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332990    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:34:39.345474 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332992    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:34:39.345474 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332995    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:34:39.345474 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.332998    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:34:39.345474 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333000    5820 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:34:39.345474 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333002    5820 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:34:39.345474 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333005    5820 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:34:39.346294 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333008    5820 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:34:39.346294 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333010    5820 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:34:39.346294 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333013    5820 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:34:39.346294 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333015    5820 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:34:39.346294 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333018    5820 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:34:39.346294 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333020    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:34:39.346294 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333023    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:34:39.346294 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333026    5820 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:34:39.346294 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333028    5820 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:34:39.346294 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333031    5820 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:34:39.346294 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333033    5820 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:34:39.346294 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333035    5820 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:34:39.346294 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333038    5820 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:34:39.346294 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333040    5820 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:34:39.346294 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333043    5820 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:34:39.346294 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333046    5820 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:34:39.346294 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333048    5820 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:34:39.346294 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333051    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:34:39.346294 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333054    5820 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:34:39.346294 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333056    5820 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:34:39.347130 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333058    5820 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:34:39.347130 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333061    5820 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:34:39.347130 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333063    5820 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:34:39.347130 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333066    5820 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:34:39.347130 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333068    5820 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:34:39.347130 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333070    5820 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:34:39.347130 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333073    5820 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:34:39.347130 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333075    5820 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:34:39.347130 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333078    5820 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:34:39.347130 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333080    5820 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:34:39.347130 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333083    5820 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:34:39.347130 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333085    5820 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:34:39.347130 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333088    5820 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:34:39.347130 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333090    5820 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:34:39.347130 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333092    5820 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:34:39.347130 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333095    5820 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:34:39.347130 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333097    5820 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:34:39.347130 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333101    5820 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:34:39.347130 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333105    5820 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:34:39.347657 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333108    5820 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:34:39.347657 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333112    5820 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:34:39.347657 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.333136    5820 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:34:39.347657 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333168    5820 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:34:39.347657 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333173    5820 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:34:39.347657 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333178    5820 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:34:39.347657 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333181    5820 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:34:39.347657 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333184    5820 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:34:39.347657 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333187    5820 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:34:39.347657 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333189    5820 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:34:39.347657 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333192    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:34:39.347657 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333194    5820 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:34:39.347657 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333196    5820 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:34:39.347657 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333199    5820 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:34:39.347657 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333201    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:34:39.348080 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333204    5820 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:34:39.348080 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333207    5820 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:34:39.348080 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333209    5820 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:34:39.348080 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333212    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:34:39.348080 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333214    5820 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:34:39.348080 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333216    5820 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:34:39.348080 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333219    5820 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:34:39.348080 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333224    5820 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:34:39.348080 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333226    5820 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:34:39.348080 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333229    5820 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:34:39.348080 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333231    5820 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:34:39.348080 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333234    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:34:39.348080 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333236    5820 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:34:39.348080 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333239    5820 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:34:39.348080 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333241    5820 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:34:39.348080 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333244    5820 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:34:39.348080 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333246    5820 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:34:39.348080 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333249    5820 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:34:39.348080 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333252    5820 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:34:39.348080 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333254    5820 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:34:39.348576 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333257    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:34:39.348576 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333260    5820 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:34:39.348576 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333264    5820 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:34:39.348576 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333269    5820 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:34:39.348576 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333272    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:34:39.348576 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333275    5820 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:34:39.348576 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333278    5820 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:34:39.348576 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333280    5820 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:34:39.348576 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333283    5820 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:34:39.348576 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333285    5820 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:34:39.348576 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333288    5820 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:34:39.348576 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333290    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:34:39.348576 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333293    5820 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:34:39.348576 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333295    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:34:39.348576 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333298    5820 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:34:39.348576 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333300    5820 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:34:39.348576 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333303    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:34:39.348576 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333305    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:34:39.348576 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333308    5820 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:34:39.349042 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333312    5820 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:34:39.349042 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333314    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:34:39.349042 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333317    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:34:39.349042 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333319    5820 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:34:39.349042 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333322    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:34:39.349042 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333325    5820 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:34:39.349042 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333327    5820 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:34:39.349042 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333330    5820 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:34:39.349042 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333332    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:34:39.349042 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333335    5820 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:34:39.349042 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333338    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:34:39.349042 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333341    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:34:39.349042 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333344    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:34:39.349042 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333346    5820 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:34:39.349042 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333349    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:34:39.349042 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333352    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:34:39.349042 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333356    5820 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:34:39.349042 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333359    5820 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:34:39.349042 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333361    5820 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:34:39.349494 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333364    5820 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:34:39.349494 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333367    5820 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:34:39.349494 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333369    5820 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:34:39.349494 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333372    5820 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:34:39.349494 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333375    5820 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:34:39.349494 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333377    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:34:39.349494 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333379    5820 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:34:39.349494 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333382    5820 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:34:39.349494 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333385    5820 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:34:39.349494 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333387    5820 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:34:39.349494 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333390    5820 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:34:39.349494 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333393    5820 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:34:39.349494 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333395    5820 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:34:39.349494 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333399    5820 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:34:39.349494 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333401    5820 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:34:39.349494 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333404    5820 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:34:39.349494 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333407    5820 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:34:39.349494 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333409    5820 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:34:39.349494 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333411    5820 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:34:39.349494 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333414    5820 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:34:39.350033 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333417    5820 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:34:39.350033 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333419    5820 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:34:39.350033 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333422    5820 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:34:39.350033 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333425    5820 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:34:39.350033 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333427    5820 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:34:39.350033 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333430    5820 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:34:39.350033 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333434    5820 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:34:39.350033 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333436    5820 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:34:39.350033 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333439    5820 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:34:39.350033 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333442    5820 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:34:39.350033 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333446    5820 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:34:39.350033 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333448    5820 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:34:39.350033 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333451    5820 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:34:39.350033 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333453    5820 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:34:39.350033 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333456    5820 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:34:39.350033 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:39.333458    5820 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:34:39.350033 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.333515    5820 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 18:34:39.350033 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.334492    5820 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 18:34:39.350033 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.334544    5820 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 18:34:39.350484 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.334725    5820 server.go:1014] "Starting client certificate rotation"
Oct 30 18:34:39.350484 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.334878    5820 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 18:34:39.350484 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.334924    5820 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:28:26 +0000 UTC" deadline="2028-07-08 05:35:46.68437396 +0000 UTC"
Oct 30 18:34:39.350484 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.334976    5820 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14795h1m7.349400119s"
Oct 30 18:34:39.350484 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.335235    5820 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:34:39.350484 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.337573    5820 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:34:39.350484 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.339113    5820 log.go:25] "Validated CRI v1 runtime API"
Oct 30 18:34:39.350484 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.341569    5820 log.go:25] "Validated CRI v1 image API"
Oct 30 18:34:39.350484 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.342373    5820 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 18:34:39.350484 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.344647    5820 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b5bc3841-233d-4f6b-9a64-6dcb4c75c1c8:/dev/nvme0n1p3 f3278fdb-3dec-4bc4-9455-90812e3d7be6:/dev/nvme0n1p4]
Oct 30 18:34:39.350771 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.344675    5820 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/00505abb739fe1fb635c5bef9337ca21aceb3738ce32d4ed32a584a25f1a2bb2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/00505abb739fe1fb635c5bef9337ca21aceb3738ce32d4ed32a584a25f1a2bb2/userdata/shm major:0 minor:62 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0391a54f7e105251436e4777f9ef3d0bf9de145bfdf57e6d4791f0e2de41f9c5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0391a54f7e105251436e4777f9ef3d0bf9de145bfdf57e6d4791f0e2de41f9c5/userdata/shm major:0 minor:358 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0515afe254d6c0481584c8483f6a489eca877016a7560058b6a645a864c7534a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0515afe254d6c0481584c8483f6a489eca877016a7560058b6a645a864c7534a/userdata/shm major:0 minor:412 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0590af5ae74b4a40ae48bbd883a43b993ae845015c639ef0b857b46f368291ea/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0590af5ae74b4a40ae48bbd883a43b993ae845015c639ef0b857b46f368291ea/userdata/shm major:0 minor:286 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1cb10ae0ef4e8e9ef9894188e1ce202d8ac2cfdd80afa785ee39fd1bab7d5265/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1cb10ae0ef4e8e9ef9894188e1ce202d8ac2cfdd80afa785ee39fd1bab7d5265/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/20eb7acf3821a2acf3f28ee6fde8dd72950a5c8a215d3e8d6c9bf99861bb6720/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/20eb7acf3821a2acf3f28ee6fde8dd72950a5c8a215d3e8d6c9bf99861bb6720/userdata/shm major:0 minor:53 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/212e735c50505ff6ea0b59b17796bc0e130cfb49f55ca39ae1055e8ada6c7223/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/212e735c50505ff6ea0b59b17796bc0e130cfb49f55ca39ae1055e8ada6c7223/userdata/shm major:0 minor:52 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/22955d982cb91e10acdb17253bb5869feb48742405a22e2aebe0946d82f756ba/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/22955d982cb91e10acdb17253bb5869feb48742405a22e2aebe0946d82f756ba/userdata/shm major:0 minor:360 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2b147797072e843fa03d0d71237c9e289b35e26cebcdd81b9a3b7ff26388ca13/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2b147797072e843fa03d0d71237c9e289b35e26cebcdd81b9a3b7ff26388ca13/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3a44400ad69e02975e061a9f803ace91f50a86644a2df50b0c7040ec8f11aebd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3a44400ad69e02975e061a9f803ace91f50a86644a2df50b0c7040ec8f11aebd/userdata/shm major:0 minor:89 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/43cd4cc8435d93fd0c9c87d3f00169ba3775f56ea3e2eafdadb56403d61fc493/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/43cd4cc8435d93fd0c9c87d3f00169ba3775f56ea3e2eafdadb56403d61fc493/userdata/shm major:0 minor:268 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/462aa255cefeccaa7502195b87d7c8ef1d8bef34ec5e20ba24d5e2703aec945d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/462aa255cefeccaa7502195b87d7c8ef1d8bef34ec5e20ba24d5e2703aec945d/userdata/shm major:0 minor:249 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/48aec6da0f776fb642caeda9775dc3cb704583591bc8a7d7fe453b8c84663736/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/48aec6da0f776fb642caeda9775dc3cb704583591bc8a7d7fe453b8c84663736/userdata/shm major:0 minor:399 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/514e0a6bd3f70c8d6839d7c8b54472ea7601330ae334354af1c1c1f3293f031c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/514e0a6bd3f70c8d6839d7c8b54472ea7601330ae334354af1c1c1f3293f031c/userdata/shm major:0 minor:291 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/55118ad4b9c2f2293db93c7fd1ee53ba589d426955efe3c0aca30e77459d24c9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/55118ad4b9c2f2293db93c7fd1ee53ba589d426955efe3c0aca30e77459d24c9/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/58b00c4b2632f708912066d204eb2308f9064bb14376596a20357463a81951c7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/58b00c4b2632f708912066d204eb2308f9064bb14376596a20357463a81951c7/userdata/shm major:0 minor:356 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6855c78ac71ded17d8193720a1e1dd2bddeab009c2df60b0f48db65b03d540c2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6855c78ac71ded17d8193720a1e1dd2bddeab009c2df60b0f48db65b03d540c2/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6f003228b1d30cf5260857ba5f58754dc25cc6b038c6d90fcfab7263e4b95825/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6f003228b1d30cf5260857ba5f58754dc25cc6b038c6d90fcfab7263e4b95825/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/79a1052e4b6242529ce9541bfff5f9b476b8f9750cf6e6de0732ccfc3b54ceb3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/79a1052e4b6242529ce9541bfff5f9b476b8f9750cf6e6de0732ccfc3b54ceb3/userdata/shm major:0 minor:294 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/81477958f53c63164f8c06fe47dbf411f367635a5caedb46f7fb6d02c7b42abb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/81477958f53c63164f8c06fe47dbf411f367635a5caedb46f7fb6d02c7b42abb/userdata/shm major:0 minor:63 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8eba0dd7e02915db6673073d0511c7ef49110c853ecfc4e90032da3eeb645118/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8eba0dd7e02915db6673073d0511c7ef49110c853ecfc4e90032da3eeb645118/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b1e40d3592a73c7b18b1a77643b79ec547e87f7468511e27bab38a1ba707246d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b1e40d3592a73c7b18b1a77643b79ec547e87f7468511e27bab38a1ba707246d/userdata/shm major:0 minor:340 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e24685c45cd1c1f042e3505248a05f1579704e97d3d9750ca6a6bcd82611622a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e24685c45cd1c1f042e3505248a05f1579704e97d3d9750ca6a6bcd82611622a/userdata/shm major:0 minor:391 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f1248a0b7920156836ee332eae6b906552c53e8e532641c6e807aa6e3399b40c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f1248a0b7920156836ee332eae6b906552c53e8e532641c6e807aa6e3399b40c/userdata/shm major:0 minor:371 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f5a1c67be6c1aecd39285c7c43774217e0fb6a8d6640de48c8262494d7f8c4b2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f5a1c67be6c1aecd39285c7c43774217e0fb6a8d6640de48c8262494d7f8c4b2/userdata/shm major:0 minor:77 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f942e7ffa5a1f0246723ff144901e3c94c2e3616b065473354c0ef5804b08343/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f942e7ffa5a1f0246723ff144901e3c94c2e3616b065473354c0ef5804b08343/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/08a8e709-63af-4d71-b084-cf533ea620e5/volumes/kubernetes.io~projected/kube-api-access-25k6c:{mountpoint:/var/lib/kubelet/pods/08a8e709-63af-4d71-b084-cf533ea620e5/volumes/kubernetes.io~projected/kube-api-access-25k6c major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/08a8e709-63af-4d71-b084-cf533ea620e5/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/08a8e709-63af-4d71-b084-cf533ea620e5/volumes/kubernetes.io~secret/metrics-certs major:0 minor:345 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0c816c10-1c32-4538-8724-834ebb405960/volumes/kubernetes.io~projected/kube-api-access-zl92s:{mountpoint:/var/lib/kubelet/pods/0c816c10-1c32-4538-8724-834ebb405960/volumes/kubernetes.io~projected/kube-api-access-zl92s major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1538afd4-e5de-4372-aa9c-895efc60d643/volumes/kubernetes.io~projected/kube-api-access-sbhr8:{mountpoint:/var/lib/kubelet/pods/1538afd4-e5de-4372-aa9c-895efc60d643/volumes/kubernetes.io~projected/kube-api-access-sbhr8 major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1538afd4-e5de-4372-aa9c-895efc60d643/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/1538afd4-e5de-4372-aa9c-895efc60d643/volumes/kubernetes.io~secret/cert major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/220393aa-93ee-4113-b373-ecd27198f170/volumes/kubernetes.io~projected/kube-api-access-j7rqz:{mountpoint:/var/lib/kubelet/pods/220393aa-93ee-4113-b373-ecd27198f170/volumes/kubernetes.io~projected/kube-api-access-j7rqz major:0 minor:355 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/220393aa-93ee-4113-b373-ecd27198f170/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/220393aa-93ee-4113-b373-ecd27198f170/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:353 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/220393aa-93ee-4113-b373-ecd27198f170/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/220393aa-93ee-4113-b373-ecd27198f170/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:352 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/220393aa-93ee-4113-b373-ecd27198f170/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/220393aa-93ee-4113-b373-ecd27198f170/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:349 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/220393aa-93ee-4113-b373-ecd27198f170/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/220393aa-93ee-4113-b373-ecd27198f170/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:351 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/220393aa-93ee-4113-b373-ecd27198f170/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/220393aa-93ee-4113-b373-ecd27198f170/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:350 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/220393aa-93ee-4113-b373-ecd27198f170/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/220393aa-93ee-4113-b373-ecd27198f170/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:343 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2ae0bc76-971d-4020-972f-6fb355e71aa0/volumes/kubernetes.io~projected/kube-api-access-5djbl:{mountpoint:/var/lib/kubelet/pods/2ae0bc76-971d-4020-972f-6fb355e71aa0/volumes/kubernetes.io~projected/kube-api-access-5djbl major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2ae0bc76-971d-4020-972f-6fb355e71aa0/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/2ae0bc76-971d-4020-972f-6fb355e71aa0/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3443d97d-774b-4df3-abe3-3ca2d6b574b6/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/3443d97d-774b-4df3-abe3-3ca2d6b574b6/volumes/kubernetes.io~secret/agent-certs major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84/volumes/kubernetes.io~projected/kube-api-access-7d8jv:{mountpoint:/var/lib/kubelet/pods/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84/volumes/kubernetes.io~projected/kube-api-access-7d8jv major:0 minor:246 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:245 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:241 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6300215e-2302-47ed-bf47-05a165277c08/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/6300215e-2302-47ed-bf47-05a165277c08/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:344 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7bfc481b-9824-43c5-acf0-436641114b03/volumes/kubernetes.io~projected/kube-api-access-5g5sn:{mountpoint:/var/lib/kubelet/pods/7bfc481b-9824-43c5-acf0-436641114b03/volumes/kubernetes.io~projected/kube-api-access-5g5sn major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/824f84de-51ea-4435-9aee-9fbcde961d41/volumes/kubernetes.io~projected/kube-api-access-t4z99:{mountpoint:/var/lib/kubelet/pods/824f84de-51ea-4435-9aee-9fbcde961d41/volumes/kubernetes.io~projected/kube-api-access-t4z99 major:0 minor:411 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/824f84de-51ea-4435-9aee-9fbcde961d41/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/824f84de-51ea-4435-9aee-9fbcde961d41/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:410 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/824f84de-51ea-4435-9aee-9fbcde961d41/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/824f84de-51ea-4435-9aee-9fbcde961d41/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:409 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8876fbb5-cdb6-4755-9122-8640d5c86560/volumes/kubernetes.io~projected/kube-api-access-km4hl:{mountpoint:/var/lib/kubelet/pods/8876fbb5-cdb6-4755-9122-8640d5c86560/volumes/kubernetes.io~projected/kube-api-access-km4hl major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8876fbb5-cdb6-4755-9122-8640d5c86560/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/8876fbb5-cdb6-4755-9122-8640d5c86560/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:289 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8876fbb5-cdb6-4755-9122-8640d5c86560/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/8876fbb5-cdb6-4755-9122-8640d5c86560/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a/volumes/kubernetes.io~projected/kube-api-access-ms9kr:{mountpoint:/var/lib/kubelet/pods/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a/volumes/kubernetes.io~projected/kube-api-access-ms9kr major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a864943c-d7eb-4486-99a6-7961573b57d4/volumes/kubernetes.io~projected/kube-api-access-zdl78:{mountpoint:/var/lib/kubelet/pods/a864943c-d7eb-4486-99a6-7961573b57d4/volumes/kubernetes.io~projected/kube-api-access-zdl78 major:0 minor:354 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a9c43ecd-b447-4e07-95cf-01523d71dda9/volumes/kubernetes.io~projected/kube-api-access-l2r2b:{mountpoint:/var/lib/kubelet/pods/a9c43ecd-b447-4e07-95cf-01523d71dda9/volumes/kubernetes.io~projected/kube-api-access-l2r2b major:0 minor:283 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a9c43ecd-b447-4e07-95cf-01523d71dda9/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/a9c43ecd-b447-4e07-95cf-01523d71dda9/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:282 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a9c43ecd-b447-4e07-95cf-01523d71dda9/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/a9c43ecd-b447-4e07-95cf-01523d71dda9/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:277 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b1735618-e0f6-49b1-ade6-8fef300791a1/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/b1735618-e0f6-49b1-ade6-8fef300791a1/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b1735618-e0f6-49b1-ade6-8fef300791a1/volumes/kubernetes.io~projected/kube-api-access-2zzhj:{mountpoint:/var/lib/kubelet/pods/b1735618-e0f6-49b1-ade6-8fef300791a1/volumes/kubernetes.io~projected/kube-api-access-2zzhj major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b1735618-e0f6-49b1-ade6-8fef300791a1/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/b1735618-e0f6-49b1-ade6-8fef300791a1/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf17e11d-1428-47c8-9aea-9f6b7c831eb1/volumes/kubernetes.io~projected/kube-api-access-4rlnr:{mountpoint:/var/lib/kubelet/pods/cf17e11d-1428-47c8-9aea-9f6b7c831eb1/volumes/kubernetes.io~projected/kube-api-access-4rlnr major:0 minor:393 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf17e11d-1428-47c8-9aea-9f6b7c831eb1/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/cf17e11d-1428-47c8-9aea-9f6b7c831eb1/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:386 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf17e11d-1428-47c8-9aea-9f6b7c831eb1/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/cf17e11d-1428-47c8-9aea-9f6b7c831eb1/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:388 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf17e11d-1428-47c8-9aea-9f6b7c831eb1/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/cf17e11d-1428-47c8-9aea-9f6b7c831eb1/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:387 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d22b0cdb-141d-4710-9f1f-90797d952a54/volumes/kubernetes.io~projected/kube-api-access-srphr:{mountpoint:/var/lib/kubelet/pods/d22b0cdb-141d-4710-9f1f-90797d952a54/volumes/kubernetes.io~projected/kube-api-access-srphr major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d22b0cdb-141d-4710-9f1f-90797d952a54/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/d22b0cdb-141d-4710-9f1f-90797d952a54/volumes/kubernetes.io~secret/metrics-tls major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d3fa116f-e372-4a36-9a57-4832dc1440b3/volumes/kubernetes.io~projected/kube-api-access-99psk:{mountpoint:/var/lib/kubelet/pods/d3fa116f-e372-4a36-9a57-4832dc1440b3/volumes/kubernetes.io~projected/kube-api-access-99psk major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d4e65089-4bdd-431f-b338-1596fbb4b02b/volumes/kubernetes.io~projected/kube-api-access-tlwnx:{mountpoint:/var/lib/kubelet/pods/d4e65089-4bdd-431f-b338-1596fbb4b02b/volumes/kubernetes.io~projected/kube-api-access-tlwnx major:0 minor:266 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d4e65089-4bdd-431f-b338-1596fbb4b02b/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/d4e65089-4bdd-431f-b338-1596fbb4b02b/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:257 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d4e65089-4bdd-431f-b338-1596fbb4b02b/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/d4e65089-4bdd-431f-b338-1596fbb4b02b/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:258 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dc7462f0-a45f-487a-99b9-3e89f783cb01/volumes/kubernetes.io~projected/kube-api-access-vj846:{mountpoint:/var/lib/kubelet/pods/dc7462f0-a45f-487a-99b9-3e89f783cb01/volumes/kubernetes.io~projected/kube-api-access-vj846 major:0 minor:385 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dc7462f0-a45f-487a-99b9-3e89f783cb01/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/dc7462f0-a45f-487a-99b9-3e89f783cb01/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:384 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d/volumes/kubernetes.io~empty-dir/tmp major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d/volumes/kubernetes.io~projected/kube-api-access-vtpk7:{mountpoint:/var/lib/kubelet/pods/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d/volumes/kubernetes.io~projected/kube-api-access-vtpk7 major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e4104cc2-a8a5-4985-83cd-e593bf12a68f/volumes/kubernetes.io~projected/kube-api-access-42nvt:{mountpoint:/var/lib/kubelet/pods/e4104cc2-a8a5-4985-83cd-e593bf12a68f/volumes/kubernetes.io~projected/kube-api-access-42nvt major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e4104cc2-a8a5-4985-83cd-e593bf12a68f/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/e4104cc2-a8a5-4985-83cd-e593bf12a68f/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:285 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e4104cc2-a8a5-4985-83cd-e593bf12a68f/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/e4104cc2-a8a5-4985-83cd-e593bf12a68f/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:284 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd/volumes/kubernetes.io~projected/kube-api-access-cm96j:{mountpoint:/var/lib/kubelet/pods/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd/volumes/kubernetes.io~projected/kube-api-access-cm96j major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:321 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volume-subpaths/web-config/alertmanager/9 major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volume-subpaths/web-config/config-reloader/6 major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~empty-dir/config-out major:0 minor:326 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~projected/kube-api-access-d4bcf:{mountpoint:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~projected/kube-api-access-d4bcf major:0 minor:334 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~projected/tls-assets major:0 minor:324 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:321 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~secret/config-volume major:0 minor:319 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:325 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:315 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:320 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:323 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~secret/web-config major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb0fef6c-8c89-40af-bd03-99cb7035a3f2/volumes/kubernetes.io~projected/kube-api-access-px2j7:{mountpoint:/var/lib/kubelet/pods/fb0fef6c-8c89-40af-bd03-99cb7035a3f2/volumes/kubernetes.io~projected/kube-api-access-px2j7 major:0 minor:51 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/8cf2e24290866e5ef89b193030b3d887d788b57a077f958f7fd47c0ddf86aa0c/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/3f338427eb187027d74eacd8c9de451c0634598b443442a94697a8e270d315f1/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/85373d111bc8578b2bd3ee192bdedbe0133b285ffeb612b1a9436b68944d1784/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/d28b8f882e2d18570e82e737edddca382f9737583fd787276638ea638200553a/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/1767c12cd1ad3eea87f370eb13ef29549cde52142f61ce8845d5486adae115da/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/e80ed8ca4812b226f39361af3effb9a89b389c9f0c05d74b3cc72254c8dae334/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/8162d32f2d5b183fa1dab0b04b4718868ced7db44006e336cb54f31e4edbd75e/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/41b8a0127089486e4666f412666257d932a6541820f8aba1223d34a38c322dd7/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/c5d2dc7af7d68781551dd851480d873458ad16714618022634c3fda14d7e0ea2/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/9ec551dc45c64486225060486f04597806cad64f0b52f1ab85f480fc96b5300b/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-136:{mountpoint:/var/lib/containers/storage/overlay/fede111f8b994c3f32f40b52a780b9b808c6db086892757c6b2d63e80a7968af/merged major:0 minor:136 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/01ddd67d63fe6610c16338f47db634693e80388c8d965e20f6d3f175fb3243b1/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/d044efc0ee14de91690c728e492ebea2dfc41287307ebd9640a15be63eba3e81/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/56387d8a2500ab03bdf7b76ab3e0da08442d2fa30e1219fad505741b9244440f/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-159:{mountpoint:/var/lib/containers/storage/overlay/1d374b83265666c67c47f9e6cc41c6491437e016514ef6a7f3f48acae38f952d/merged major:0 minor:159 fsType:overlay blockSize:0} overlay_0-161:{mountpoint:/var/lib/containers/storage/overlay/9c04d202b2bdc033d1b34a628edb605c602fad9b17795cbf2782ca8bf6fdc502/merged major:0 minor:161 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/2271100c269f5be292ff5124f6b651debbaf32abbcea4db4579d671c90e8b2a6/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/d07372b17526a7df67e8b5fd5cbe9c69fb5c54e2960d4b3d7f74977f841f8152/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/708c73c838ec35bbb431c9cc7eef33228ec70b4a9f21efe1815f2872215baee3/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-184:{mountpoint:/var/lib/containers/storage/overlay/784886ea9a9d6c7105424232334ffede95b8e666427635f88eb57d0acf6d6850/merged major:0 minor:184 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/f76a4a3ebb7ce7ca6be70977ff6a728e3e333d74be088651727fcaa5e44d565e/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/3c74b25ea88c7c0bad322042a1972d2c3eaab0c96142dbb4661e3173805e46ac/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/e392a3e3bf08ee84bdc447285159aaee9dabeb62c425766bcf62ce4ac62ac023/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/98b0ad0556ad0a145e2bb42e81df5e513bf7fd42d2d1d81dbacbdb4919bbcaaa/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/65fceba31fecb99f81845757a511ec2f993da01e3daaebaaaded96e006c971b5/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-218:{mountpoint:/var/lib/containers/storage/overlay/9c6d5a39ade148813a28a5b98a3d0c719536dadf2cc7b44ddb6875c2e2f1c18c/merged major:0 minor:218 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/2082e3ea06af6b31cab9dfe5a195971e464400415b92aa2cba4fd4aae70dfc9c/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/880598c0683d4d0d59ca6d599ced9b7ac1312dd0a10858cac53a640156b9282f/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-236:{mountpoint:/var/lib/containers/storage/overlay/ffd9b22cfa89773cc9c7f06f6812d4d8185e312bca12fc5bed9293e0027bf316/merged major:0 minor:236 fsType:overlay blockSize:0} overlay_0-247:{mountpoint:/var/lib/containers/storage/overlay/22d1ad35901e3532212a70340d9fd8349905fd39d3f20df6e5b69e6a15a1d320/merged major:0 minor:247 fsType:overlay blockSize:0} overlay_0-251:{mountpoint:/var/lib/containers/storage/overlay/3a5496ef66a3259f1f9e95d7b76ba1acb73581340d5a05fd44e8ca5cc93724ff/merged major:0 minor:251 fsType:overlay blockSize:0} overlay_0-259:{mountpoint:/var/lib/containers/storage/overlay/79aad6aa1c5130c398348b597878fc08c6b0f8d7103cc7a1f8f57a9ab8c4d838/merged major:0 minor:259 fsType:overlay blockSize:0} overlay_0-261:{mountpoint:/var/lib/containers/storage/overlay/89b12407989ff19df8d1a9dd6f2b1f029bc715f2573bf1a3f9076db78af6e910/merged major:0 minor:261 fsType:overlay blockSize:0} overlay_0-273:{mountpoint:/var/lib/containers/storage/overlay/7d6927255a3bf1ecbee812e9d5aa73555a477da605daa82cb46bc779fc869a3f/merged major:0 minor:273 fsType:overlay blockSize:0} overlay_0-275:{mountpoint:/var/lib/containers/storage/overlay/2e969fd84925e62ede27d3aa3bb3cc3023966bf8fd86fd0aed6594d99fdb04ee/merged major:0 minor:275 fsType:overlay blockSize:0} overlay_0-296:{mountpoint:/var/lib/containers/storage/overlay/36bb7ca5da40c6457a017493f07e871220c39a7d8550c73d19a30bdf6498981f/merged major:0 minor:296 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/2f802b8dce0353c6e41a1486dab8d8386f13002e86167b9fd4df70a362465d05/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/9aada431b39d1cdcf3c51686bfa6f00615e99cc4da586ab45a23fd902cdcf22d/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/9b838d89cd0bca36a2caa5658e11a7ab9f03d949762cb8e7da13241f17f0de0a/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-308:{mountpoint:/var/lib/containers/storage/overlay/fd9bd9a3e1b13480414e6a9b9655a1bff5707ed0e7e284ddf62923f083256caf/merged major:0 minor:308 fsType:overlay blockSize:0} overlay_0-312:{mountpoint:/var/lib/containers/storage/overlay/c64afe8386c3653377286e2a8b3de650bb418b429b2c641664fff7c0f3ef6558/merged major:0 minor:312 fsType:overlay blockSize:0} overlay_0-327:{mountpoint:/var/lib/containers/storage/overlay/f52faf110ee54f38e391d3316109f344f49e6157b21a3eba1006267d7b7a72d4/merged major:0 minor:327 fsType:overlay blockSize:0} overlay_0-329:{mountpoint:/var/lib/containers/storage/overlay/efce3ac5e013e75fd69aec8b0d85e787866283ce86b684a30795d1f8494df4de/merged major:0 minor:329 fsType:overlay blockSize:0} overlay_0-336:{mountpoint:/var/lib/containers/storage/overlay/3b9b28ff7a307b1c9d5ba08909a324235120b765758cceef548e18ed4d968dd7/merged major:0 minor:336 fsType:overlay blockSize:0} overlay_0-362:{mountpoint:/var/lib/containers/storage/overlay/3d56a0b0636e226ccb40c66e0cb5dc135310bb97c1668e3d28332511ec60fb9c/merged major:0 minor:362 fsType:overlay blockSize:0} overlay_0-364:{mountpoint:/var/lib/containers/storage/overlay/f487025fdf9e8a16239fd78782b3156d49767f0eb91f7f90553dcb0868747e28/merged major:0 minor:364 fsType:overlay blockSize:0} overlay_0-368:{mountpoint:/var/lib/containers/storage/overlay/fd5ab3df9e9cb289ce6b93a563deaa65ceb27ca912b67873bcd7c46cb53578de/merged major:0 minor:368 fsType:overlay blockSize:0} overlay_0-376:{mountpoint:/var/lib/containers/storage/overlay/ccec4502291029cf96cd24b0af19804af32a7760fa683e8d312ff490a5851ea9/merged major:0 minor:376 fsType:overlay blockSize:0} overlay_0-378:{mountpoint:/var/lib/containers/storage/overlay/5738fe8937a1956aeaf226937c54a95cefe8cf189952e0b2633191dd2255d492/merged major:0 minor:378 fsType:overlay blockSize:0} overlay_0-380:{mountpoint:/var/lib/containers/storage/overlay/2be26ba7a11f582b63ec62e6e40b4e7c5efa1d1dd3c979bd9975e04d7bb25266/merged major:0 minor:380 fsType:overlay blockSize:0} overlay_0-382:{mountpoint:/var/lib/containers/storage/overlay/10ef274bd4a2f4bb761cd24fba9b1a3f20a726aa3107e0e94f22640599fb35d8/merged major:0 minor:382 fsType:overlay blockSize:0} overlay_0-389:{mountpoint:/var/lib/containers/storage/overlay/84bc4b1fba9257d0338da2131ae91d2d6786255cf296de76d5a1bd716eaf3529/merged major:0 minor:389 fsType:overlay blockSize:0} overlay_0-394:{mountpoint:/var/lib/containers/storage/overlay/bd3a85f4fbe110e960329af1aabb194021bbadd2c2282a53e3744b6f5bcdd3e8/merged major:0 minor:394 fsType:overlay blockSize:0} overlay_0-402:{mountpoint:/var/lib/containers/storage/overlay/c54891adc23540e7061c46ef5828e90aa0aecf319e3ec72e40534b02391b93f1/merged major:0 minor:402 fsType:overlay blockSize:0} overlay_0-407:{mountpoint:/var/lib/containers/storage/overlay/20b82d0cd9ed0a09c275ca48592b902a24ab1d83c651f6ec474ee8e6e37764d6/merged major:0 minor:407 fsType:overlay blockSize:0} overlay_0-414:{mountpoint:/var/lib/containers/storage/overlay/c1998a67c43dc746329994b2ffa3926b67a133bed05fdd7fdca08cda04c60df8/merged major:0 minor:414 fsType:overlay blockSize:0} overlay_0-416:{mountpoint:/var/lib/containers/storage/overlay/bd462102342dbcd82b42efb2c6ad5a5b6d6ac8fbef4b03c38ba5751492ec59c6/merged major:0 minor:416 fsType:overlay blockSize:0} overlay_0-418:{mountpoint:/var/lib/containers/storage/overlay/bba927d00b2b45d39d82cb88f266c5bd818f2bbafa5182c6cee874260423e852/merged major:0 minor:418 fsType:overlay blockSize:0} overlay_0-420:{mountpoint:/var/lib/containers/storage/overlay/e12b90c8f8db3aa31eb43174a5838df25798398f685758772f0ce2a77350d904/merged major:0 minor:420 fsType:overlay blockSize:0} overlay_0-430:{mountpoint:/var/lib/containers/storage/overlay/95bfec244971d2ac6ebfe92279fd90591f043d89d3e15f43fc4207ba7e618748/merged major:0 minor:430 fsType:overlay blockSize:0} overlay_0-435:{mountpoint:/var/lib/containers/storage/overlay/866aa967ff380db77b998bd28b6093dc990de4cf86a5c7c1df3b4f8a2d8566cd/merged major:0 minor:435 fsType:overlay blockSize:0} overlay_0-440:{mountpoint:/var/lib/containers/storage/overlay/8be3fbb8e778785bc0bfc144588eaba79ecae1fe99dc5a341b3d29e59f734ab1/merged major:0 minor:440 fsType:overlay blockSize:0} overlay_0-446:{mountpoint:/var/lib/containers/storage/overlay/3405eb1c8497df6a6bf9f9f7b1a8be83064092ed4b6eaee5e867c7b4bbc34137/merged major:0 minor:446 fsType:overlay blockSize:0} overlay_0-451:{mountpoint:/var/lib/containers/storage/overlay/d626be30bf86fad0db76c84d556da7f0dfbf57f25a72a8c88a18789cfe0a82ae/merged major:0 minor:451 fsType:overlay blockSize:0} overlay_0-456:{mountpoint:/var/lib/containers/storage/overlay/d2d350e5df41111de428592b2d92dda626e5029285536575f48e7337d026024e/merged major:0 minor:456 fsType:overlay blockSize:0} overlay_0-461:{mountpoint:/var/lib/containers/storage/overlay/d662e42ee1804ec58e6ddcff22e9ca3baca7721d7cc5f8fae10dc45cffc9bfad/merged major:0 minor:461 fsType:overlay blockSize:0} overlay_0-466:{mountpoint:/var/lib/containers/storage/overlay/0bd60f82ffe6fe0dba78ca2d458bd41a5a68f37b1aabb566bc781ae119737bba/merged major:0 minor:466 fsType:overlay blockSize:0} overlay_0-472:{mountpoint:/var/lib/containers/storage/overlay/760c2ccb106d4bd37e72f6f1756aee08fd1bedd2bb3197a09872f62c6aba4e95/merged major:0 minor:472 fsType:overlay blockSize:0} overlay_0-477:{mountpoint:/var/lib/containers/storage/overlay/5927d324d55ce82a96cb53fe58f58b659676d9a3bc4c0f1362bbe3acdd9b7093/merged major:0 minor:477 fsType:overlay blockSize:0} overlay_0-481:{mountpoint:/var/lib/containers/storage/overlay/f36e13be203e3ab6aca41abe97abb3e4f9d0cad09878f203dfa1131c404757be/merged major:0 minor:481 fsType:overlay blockSize:0} overlay_0-489:{mountpoint:/var/lib/containers/storage/overlay/841e47004e96b7b8cc36207be668c8b882b1bd6cbfbbbe41dd8474f6f75a3e81/merged major:0 minor:489 fsType:overlay blockSize:0} overlay_0-493:{mountpoint:/var/lib/containers/storage/overlay/ddc7dc83b11c77cc0fc6c967a1909ed041486e76f65d0609f2cd87074a15515d/merged major:0 minor:493 fsType:overlay blockSize:0} overlay_0-499:{mountpoint:/var/lib/containers/storage/overlay/eaa9f8c65291245f0da5b05f4b65c7f9ae6a5bce637764fdd780107eed98685b/merged major:0 minor:499 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/d3c790e49fc86261c2aea782864846d04782d3daa2cc08f9593f8553cb6f5338/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/94261ec5d222555dc612ec9a95a23f30c689c1f97df01efac9f7bce2c6163a70/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/7b0bd6913c77201fdf2a45b57032c0a4636cb7154824742ec44ffc5cfd5d15c1/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/9644e796e47b78c8f02c05028d43f583c2af43596fce541c41c167681df05a8d/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/95a69e9ea41b43b7fee7dc746b15de8fd716c706525bd79949bfb7495c43dc67/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/6cfbf2451f80a9527e168834e8809dc8467a34eaeb632fbcdab53a0dedcde3e9/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/0050fca837005043c1794fb82e646a5de2c02dd577f825b810749ce46e146e6f/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/3b23a7459b26b8d375a6462a8e603583ca2d7897ecf7aaaab2a8bd0a9f3194e4/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/6f86d8c768c0208f3b069e81d90777a84d25e008359fe78e8981712bc3de6d09/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/bd01df969af0fc2fabf8076faa3316a59711fdf98a9affce836143700f7381fe/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 18:34:39.359454 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.358792    5820 manager.go:217] Machine: {Timestamp:2026-10-30 18:34:39.35822136 +0000 UTC m=+0.069970048 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100221 MemoryCapacity:32812007424 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2250145727e056726470fd8e9b60fa SystemUUID:ec225014-5727-e056-7264-70fd8e9b60fa BootID:386c8f48-c81a-4b2c-8186-4f5197fd36e0 Filesystems:[{Device:/var/lib/kubelet/pods/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:49 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/3443d97d-774b-4df3-abe3-3ca2d6b574b6/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:50 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8eba0dd7e02915db6673073d0511c7ef49110c853ecfc4e90032da3eeb645118/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1cb10ae0ef4e8e9ef9894188e1ce202d8ac2cfdd80afa785ee39fd1bab7d5265/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-308 DeviceMajor:0 DeviceMinor:308 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/22955d982cb91e10acdb17253bb5869feb48742405a22e2aebe0946d82f756ba/userdata/shm DeviceMajor:0 DeviceMinor:360 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-362 DeviceMajor:0 DeviceMinor:362 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/824f84de-51ea-4435-9aee-9fbcde961d41/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:409 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2ae0bc76-971d-4020-972f-6fb355e71aa0/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:45 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:245 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-446 DeviceMajor:0 DeviceMinor:446 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-481 DeviceMajor:0 DeviceMinor:481 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d4e65089-4bdd-431f-b338-1596fbb4b02b/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:258 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/220393aa-93ee-4113-b373-ecd27198f170/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:349 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-414 DeviceMajor:0 DeviceMinor:414 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:322 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-489 DeviceMajor:0 DeviceMinor:489 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406003712 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-273 DeviceMajor:0 DeviceMinor:273 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-382 DeviceMajor:0 DeviceMinor:382 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-389 DeviceMajor:0 DeviceMinor:389 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-440 DeviceMajor:0 DeviceMinor:440 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:320 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-435 DeviceMajor:0 DeviceMinor:435 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-493 DeviceMajor:0 DeviceMinor:493 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406003712 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6f003228b1d30cf5260857ba5f58754dc25cc6b038c6d90fcfab7263e4b95825/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-261 DeviceMajor:0 DeviceMinor:261 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b1e40d3592a73c7b18b1a77643b79ec547e87f7468511e27bab38a1ba707246d/userdata/shm DeviceMajor:0 DeviceMinor:340 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/55118ad4b9c2f2293db93c7fd1ee53ba589d426955efe3c0aca30e77459d24c9/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/462aa255cefeccaa7502195b87d7c8ef1d8bef34ec5e20ba24d5e2703aec945d/userdata/shm DeviceMajor:0 DeviceMinor:249 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/cf17e11d-1428-47c8-9aea-9f6b7c831eb1/volumes/kubernetes.io~projected/kube-api-access-4rlnr DeviceMajor:0 DeviceMinor:393 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-420 DeviceMajor:0 DeviceMinor:420 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:322 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/220393aa-93ee-4113-b373-ecd27198f170/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:353 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/cf17e11d-1428-47c8-9aea-9f6b7c831eb1/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:388 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:319 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/7bfc481b-9824-43c5-acf0-436641114b03/volumes/kubernetes.io~projected/kube-api-access-5g5sn DeviceMajor:0 DeviceMinor:60 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-161 DeviceMajor:0 DeviceMinor:161 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84/volumes/kubernetes.io~projected/kube-api-access-7d8jv DeviceMajor:0 DeviceMinor:246 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/20eb7acf3821a2acf3f28ee6fde8dd72950a5c8a215d3e8d6c9bf99861bb6720/userdata/shm DeviceMajor:0 DeviceMinor:53 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a9c43ecd-b447-4e07-95cf-01523d71dda9/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:277 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/a9c43ecd-b447-4e07-95cf-01523d71dda9/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:282 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f1248a0b7920156836ee332eae6b906552c53e8e532641c6e807aa6e3399b40c/userdata/shm DeviceMajor:0 DeviceMinor:371 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-380 DeviceMajor:0 DeviceMinor:380 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-472 DeviceMajor:0 DeviceMinor:472 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-247 DeviceMajor:0 DeviceMinor:247 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d4e65089-4bdd-431f-b338-1596fbb4b02b/volumes/kubernetes.io~projected/kube-api-access-tlwnx DeviceMajor:0 DeviceMinor:266 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:321 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/a864943c-d7eb-4486-99a6-7961573b57d4/volumes/kubernetes.io~projected/kube-api-access-zdl78 DeviceMajor:0 DeviceMinor:354 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a9c43ecd-b447-4e07-95cf-01523d71dda9/volumes/kubernetes.io~projected/kube-api-access-l2r2b DeviceMajor:0 DeviceMinor:283 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e4104cc2-a8a5-4985-83cd-e593bf12a68f/volumes/kubernetes.io~projected/kube-api-access-42nvt DeviceMajor:0 DeviceMinor:288 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/6300215e-2302-47ed-bf47-05a165277c08/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:344 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-376 DeviceMajor:0 DeviceMinor:376 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-407 DeviceMajor:0 DeviceMinor:407 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6855c78ac71ded17d8193720a1e1dd2bddeab009c2df60b0f48db65b03d540c2/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/8876fbb5-cdb6-4755-9122-8640d5c86560/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:290 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-296 DeviceMajor:0 DeviceMinor:296 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-499 DeviceMajor:0 DeviceMinor:499 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2b147797072e843fa03d0d71237c9e289b35e26cebcdd81b9a3b7ff26388ca13/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-336 DeviceMajor:0 DeviceMinor:336 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/58b00c4b2632f708912066d204eb2308f9064bb14376596a20357463a81951c7/userdata/shm DeviceMajor:0 DeviceMinor:356 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e4104cc2-a8a5-4985-83cd-e593bf12a68f/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:284 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0590af5ae74b4a40ae48bbd883a43b993ae845015c639ef0b857b46f368291ea/userdata/shm DeviceMajor:0 DeviceMinor:286 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-184 DeviceMajor:0 DeviceMinor:184 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d22b0cdb-141d-4710-9f1f-90797d952a54/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:201 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~projected/kube-api-access-d4bcf DeviceMajor:0 DeviceMinor:334 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-402 DeviceMajor:0 DeviceMinor:402 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-418 DeviceMajor:0 DeviceMinor:418 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/0c816c10-1c32-4538-8724-834ebb405960/volumes/kubernetes.io~projected/kube-api-access-zl92s DeviceMajor:0 DeviceMinor:58 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3a44400ad69e02975e061a9f803ace91f50a86644a2df50b0c7040ec8f11aebd/userdata/shm DeviceMajor:0 DeviceMinor:89 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1538afd4-e5de-4372-aa9c-895efc60d643/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:202 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/1538afd4-e5de-4372-aa9c-895efc60d643/volumes/kubernetes.io~projected/kube-api-access-sbhr8 DeviceMajor:0 DeviceMinor:203 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:315 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:323 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/220393aa-93ee-4113-b373-ecd27198f170/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:350 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f5a1c67be6c1aecd39285c7c43774217e0fb6a8d6640de48c8262494d7f8c4b2/userdata/shm DeviceMajor:0 DeviceMinor:77 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-218 DeviceMajor:0 DeviceMinor:218 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:322 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-477 DeviceMajor:0 DeviceMinor:477 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-159 DeviceMajor:0 DeviceMinor:159 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:241 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0391a54f7e105251436e4777f9ef3d0bf9de145bfdf57e6d4791f0e2de41f9c5/userdata/shm DeviceMajor:0 DeviceMinor:358 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-394 DeviceMajor:0 DeviceMinor:394 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fb0fef6c-8c89-40af-bd03-99cb7035a3f2/volumes/kubernetes.io~projected/kube-api-access-px2j7 DeviceMajor:0 DeviceMinor:51 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd/volumes/kubernetes.io~projected/kube-api-access-cm96j DeviceMajor:0 DeviceMinor:59 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-259 DeviceMajor:0 DeviceMinor:259 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/48aec6da0f776fb642caeda9775dc3cb704583591bc8a7d7fe453b8c84663736/userdata/shm DeviceMajor:0 DeviceMinor:399 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-416 DeviceMajor:0 DeviceMinor:416 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b1735618-e0f6-49b1-ade6-8fef300791a1/volumes/kubernetes.io~projected/kube-api-access-2zzhj DeviceMajor:0 DeviceMinor:54 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/d3fa116f-e372-4a36-9a57-4832dc1440b3/volumes/kubernetes.io~projected/kube-api-access-99psk DeviceMajor:0 DeviceMinor:61 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e4104cc2-a8a5-4985-83cd-e593bf12a68f/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:285 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/220393aa-93ee-4113-b373-ecd27198f170/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:343 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-378 DeviceMajor:0 DeviceMinor:378 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dc7462f0-a45f-487a-99b9-3e89f783cb01/volumes/kubernetes.io~projected/kube-api-access-vj846 DeviceMajor:0 DeviceMinor:385 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/2ae0bc76-971d-4020-972f-6fb355e71aa0/volumes/kubernetes.io~projected/kube-api-access-5djbl DeviceMajor:0 DeviceMinor:197 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/514e0a6bd3f70c8d6839d7c8b54472ea7601330ae334354af1c1c1f3293f031c/userdata/shm DeviceMajor:0 DeviceMinor:291 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/cf17e11d-1428-47c8-9aea-9f6b7c831eb1/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:386 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-451 DeviceMajor:0 DeviceMinor:451 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b1735618-e0f6-49b1-ade6-8fef300791a1/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:322 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/212e735c50505ff6ea0b59b17796bc0e130cfb49f55ca39ae1055e8ada6c7223/userdata/shm DeviceMajor:0 DeviceMinor:52 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/d22b0cdb-141d-4710-9f1f-90797d952a54/volumes/kubernetes.io~projected/kube-api-access-srphr DeviceMajor:0 DeviceMinor:204 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-275 DeviceMajor:0 DeviceMinor:275 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/220393aa-93ee-4113-b373-ecd27198f170/volumes/kubernetes.io~projected/kube-api-access-j7rqz DeviceMajor:0 DeviceMinor:355 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d4e65089-4bdd-431f-b338-1596fbb4b02b/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:257 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:326 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-368 DeviceMajor:0 DeviceMinor:368 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dc7462f0-a45f-487a-99b9-3e89f783cb01/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:384 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0515afe254d6c0481584c8483f6a489eca877016a7560058b6a645a864c7534a/userdata/shm DeviceMajor:0 DeviceMinor:412 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:321 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/81477958f53c63164f8c06fe47dbf411f367635a5caedb46f7fb6d02c7b42abb/userdata/shm DeviceMajor:0 DeviceMinor:63 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8876fbb5-cdb6-4755-9122-8640d5c86560/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:289 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-312 DeviceMajor:0 DeviceMinor:312 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:324 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-364 DeviceMajor:0 DeviceMinor:364 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a/volumes/kubernetes.io~projected/kube-api-access-ms9kr DeviceMajor:0 DeviceMinor:57 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/00505abb739fe1fb635c5bef9337ca21aceb3738ce32d4ed32a584a25f1a2bb2/userdata/shm DeviceMajor:0 DeviceMinor:62 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/43cd4cc8435d93fd0c9c87d3f00169ba3775f56ea3e2eafdadb56403d61fc493/userdata/shm DeviceMajor:0 DeviceMinor:268 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-327 DeviceMajor:0 DeviceMinor:327 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cf17e11d-1428-47c8-9aea-9f6b7c831eb1/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:387 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/824f84de-51ea-4435-9aee-9fbcde961d41/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:410 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/824f84de-51ea-4435-9aee-9fbcde961d41/volumes/kubernetes.io~projected/kube-api-access-t4z99 DeviceMajor:0 DeviceMinor:411 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-461 DeviceMajor:0 DeviceMinor:461 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d/volumes/kubernetes.io~projected/kube-api-access-vtpk7 DeviceMajor:0 DeviceMinor:55 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f942e7ffa5a1f0246723ff144901e3c94c2e3616b065473354c0ef5804b08343/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-236 DeviceMajor:0 DeviceMinor:236 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-251 DeviceMajor:0 DeviceMinor:251 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8876fbb5-cdb6-4755-9122-8640d5c86560/volumes/kubernetes.io~projected/kube-api-access-km4hl DeviceMajor:0 DeviceMinor:293 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:325 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-329 DeviceMajor:0 DeviceMinor:329 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/08a8e709-63af-4d71-b084-cf533ea620e5/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:345 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/220393aa-93ee-4113-b373-ecd27198f170/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:351 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/220393aa-93ee-4113-b373-ecd27198f170/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:352 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e24685c45cd1c1f042e3505248a05f1579704e97d3d9750ca6a6bcd82611622a/userdata/shm DeviceMajor:0 DeviceMinor:391 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:47 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/b1735618-e0f6-49b1-ade6-8fef300791a1/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:48 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/08a8e709-63af-4d71-b084-cf533ea620e5/volumes/kubernetes.io~projected/kube-api-access-25k6c DeviceMajor:0 DeviceMinor:56 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/79a1052e4b6242529ce9541bfff5f9b476b8f9750cf6e6de0732ccfc3b54ceb3/userdata/shm DeviceMajor:0 DeviceMinor:294 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-430 DeviceMajor:0 DeviceMinor:430 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-136 DeviceMajor:0 DeviceMinor:136 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-456 DeviceMajor:0 DeviceMinor:456 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-466 DeviceMajor:0 DeviceMinor:466 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:0391a54f7e10525 MacAddress:f6:01:4f:99:cf:33 Speed:10000 Mtu:8901} {Name:0515afe254d6c04 MacAddress:ee:65:67:f0:8d:ef Speed:10000 Mtu:8901} {Name:0590af5ae74b4a4 MacAddress:ee:e0:28:02:a3:23 Speed:10000 Mtu:8901} {Name:1cb10ae0ef4e8e9 MacAddress:c2:af:76:62:81:49 Speed:10000 Mtu:8901} {Name:22955d982cb91e1 MacAddress:b6:b9:66:f4:fe:c6 Speed:10000 Mtu:8901} {Name:2b147797072e843 MacAddress:6e:c3:c8:47:70:33 Speed:10000 Mtu:8901} {Name:43cd4cc8435d93f MacAddress:8a:b8:f3:0c:00:6c Speed:10000 Mtu:8901} {Name:462aa255cefecca MacAddress:66:b9:8d:17:11:26 Speed:10000 Mtu:8901} {Name:48aec6da0f776fb MacAddress:0e:1c:7e:f4:e0:dd Speed:10000 Mtu:8901} {Name:514e0a6bd3f70c8 MacAddress:62:2f:4b:39:10:5f Speed:10000 Mtu:8901} {Name:55118ad4b9c2f22 MacAddress:22:e5:69:9b:d6:3d Speed:10000 Mtu:8901} {Name:58b00c4b2632f70 MacAddress:2a:d3:eb:de:dc:f2 Speed:10000 Mtu:8901} {Name:b1e40d3592a73c7 MacAddress:1a:25:f5:3c:25:10 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:2f:6d:c8:fb:65 Speed:0 Mtu:9001} {Name:br-int MacAddress:32:3c:45:b1:33:9d Speed:0 Mtu:8901} {Name:e24685c45cd1c1f MacAddress:4e:8f:5d:0d:2b:b4 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:2f:6d:c8:fb:65 Speed:0 Mtu:9001} {Name:f1248a0b7920156 MacAddress:b2:a6:8a:19:73:3d Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:72:de:61:54:10:f6 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:e6:55:95:4e:c7:e6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812007424 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 18:34:39.359454 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.359458    5820 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 18:34:39.359781 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.359511    5820 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 18:34:39.359813 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.359774    5820 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 18:34:39.360028 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.359798    5820 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-80.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 18:34:39.360083 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.360072    5820 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 18:34:39.360129 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.360117    5820 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 18:34:39.360178 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.360136    5820 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:34:39.360178 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.360160    5820 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:34:39.360314 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.360302    5820 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 18:34:39.360619 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.360607    5820 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 18:34:39.360775 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.360764    5820 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 18:34:39.360823 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.360813    5820 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 18:34:39.360853 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.360834    5820 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 18:34:39.360853 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.360849    5820 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 18:34:39.360932 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.360865    5820 kubelet.go:405] "Adding apiserver pod source"
Oct 30 18:34:39.360932 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.360877    5820 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 18:34:39.362892 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.362870    5820 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 18:34:39.363022 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.362889    5820 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:34:39.363656 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.363612    5820 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 18:34:39.366565 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.366551    5820 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 18:34:39.366565 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.366565    5820 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:34:39.366670 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.366660    5820 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 18:34:39.366858 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.366847    5820 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 18:34:39.366919 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.366876    5820 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 18:34:39.366988 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.366978    5820 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 18:34:39.367022 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.366995    5820 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 18:34:39.367022 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.367001    5820 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 18:34:39.367022 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.367007    5820 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 18:34:39.367022 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.367012    5820 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 18:34:39.367022 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.367018    5820 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 18:34:39.367022 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.367024    5820 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 18:34:39.367272 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.367030    5820 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 18:34:39.367272 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.367037    5820 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 18:34:39.367272 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.367043    5820 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 18:34:39.367272 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.367053    5820 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 18:34:39.367272 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.367063    5820 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 18:34:39.367272 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.367092    5820 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 18:34:39.367272 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.367098    5820 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 18:34:39.370446 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.370434    5820 server.go:1263] "Started kubelet"
Oct 30 18:34:39.371290 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.370506    5820 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 18:34:39.371290 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.370803    5820 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 18:34:39.371290 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.370888    5820 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 18:34:39.371290 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.371219    5820 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 18:34:39.370997 ip-10-0-133-80 systemd[1]: Started Kubernetes Kubelet.
Oct 30 18:34:39.372599 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.372043    5820 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 18:34:39.378829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.378810    5820 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 18:34:39.379650 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.378891    5820 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 18:34:39.379650 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:34:39.378977    5820 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-80.us-west-1.compute.internal\" not found"
Oct 30 18:34:39.379650 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.379106    5820 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 18:34:39.379650 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.379121    5820 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 18:34:39.379650 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.379191    5820 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.385482    5820 factory.go:55] Registering systemd factory
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.385507    5820 factory.go:223] Registration of the systemd container factory successfully
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.387514    5820 factory.go:153] Registering CRI-O factory
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.387535    5820 factory.go:223] Registration of the crio container factory successfully
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.387669    5820 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.387780    5820 factory.go:103] Registering Raw factory
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.387797    5820 manager.go:1199] Started watching for new ooms in manager
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.388205    5820 manager.go:319] Starting recovery of all containers
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389351    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7ed38b7-0997-4cd8-9da0-41093596a6a0" volumeName="kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389392    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ae0bc76-971d-4020-972f-6fb355e71aa0" volumeName="kubernetes.io/projected/2ae0bc76-971d-4020-972f-6fb355e71aa0-kube-api-access-5djbl" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389414    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="824f84de-51ea-4435-9aee-9fbcde961d41" volumeName="kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-console-config" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389427    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="824f84de-51ea-4435-9aee-9fbcde961d41" volumeName="kubernetes.io/projected/824f84de-51ea-4435-9aee-9fbcde961d41-kube-api-access-t4z99" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389445    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1735618-e0f6-49b1-ade6-8fef300791a1" volumeName="kubernetes.io/projected/b1735618-e0f6-49b1-ade6-8fef300791a1-kube-api-access-2zzhj" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389459    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf17e11d-1428-47c8-9aea-9f6b7c831eb1" volumeName="kubernetes.io/projected/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-kube-api-access-4rlnr" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389478    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d22b0cdb-141d-4710-9f1f-90797d952a54" volumeName="kubernetes.io/projected/d22b0cdb-141d-4710-9f1f-90797d952a54-kube-api-access-srphr" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389491    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d3fa116f-e372-4a36-9a57-4832dc1440b3" volumeName="kubernetes.io/projected/d3fa116f-e372-4a36-9a57-4832dc1440b3-kube-api-access-99psk" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389510    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e4104cc2-a8a5-4985-83cd-e593bf12a68f" volumeName="kubernetes.io/empty-dir/e4104cc2-a8a5-4985-83cd-e593bf12a68f-volume-directive-shadow" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389530    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="220393aa-93ee-4113-b373-ecd27198f170" volumeName="kubernetes.io/empty-dir/220393aa-93ee-4113-b373-ecd27198f170-tmp" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389543    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8876fbb5-cdb6-4755-9122-8640d5c86560" volumeName="kubernetes.io/secret/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-tls" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389561    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf17e11d-1428-47c8-9aea-9f6b7c831eb1" volumeName="kubernetes.io/secret/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-secret-metrics-server-client-certs" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389574    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dc7462f0-a45f-487a-99b9-3e89f783cb01" volumeName="kubernetes.io/projected/dc7462f0-a45f-487a-99b9-3e89f783cb01-kube-api-access-vj846" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389587    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e5c45faf-cafe-4ee2-9faa-f8686b13c4bd" volumeName="kubernetes.io/projected/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-kube-api-access-cm96j" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389610    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7ed38b7-0997-4cd8-9da0-41093596a6a0" volumeName="kubernetes.io/configmap/f7ed38b7-0997-4cd8-9da0-41093596a6a0-metrics-client-ca" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389630    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84" volumeName="kubernetes.io/projected/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-kube-api-access-7d8jv" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389642    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a" volumeName="kubernetes.io/projected/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-kube-api-access-ms9kr" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389656    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4e65089-4bdd-431f-b338-1596fbb4b02b" volumeName="kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-service-ca" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389674    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4e65089-4bdd-431f-b338-1596fbb4b02b" volumeName="kubernetes.io/projected/d4e65089-4bdd-431f-b338-1596fbb4b02b-kube-api-access-tlwnx" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389686    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4e65089-4bdd-431f-b338-1596fbb4b02b" volumeName="kubernetes.io/secret/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-oauth-config" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389703    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e4104cc2-a8a5-4985-83cd-e593bf12a68f" volumeName="kubernetes.io/secret/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389715    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7ed38b7-0997-4cd8-9da0-41093596a6a0" volumeName="kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389727    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb0fef6c-8c89-40af-bd03-99cb7035a3f2" volumeName="kubernetes.io/projected/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-kube-api-access-px2j7" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389744    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0c816c10-1c32-4538-8724-834ebb405960" volumeName="kubernetes.io/projected/0c816c10-1c32-4538-8724-834ebb405960-kube-api-access-zl92s" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389759    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6300215e-2302-47ed-bf47-05a165277c08" volumeName="kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389776    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d22b0cdb-141d-4710-9f1f-90797d952a54" volumeName="kubernetes.io/empty-dir/d22b0cdb-141d-4710-9f1f-90797d952a54-tmp-dir" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389788    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dc7462f0-a45f-487a-99b9-3e89f783cb01" volumeName="kubernetes.io/secret/dc7462f0-a45f-487a-99b9-3e89f783cb01-monitoring-plugin-cert" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389801    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="220393aa-93ee-4113-b373-ecd27198f170" volumeName="kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389827    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3443d97d-774b-4df3-abe3-3ca2d6b574b6" volumeName="kubernetes.io/secret/3443d97d-774b-4df3-abe3-3ca2d6b574b6-agent-certs" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389847    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84" volumeName="kubernetes.io/configmap/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-metrics-client-ca" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389862    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="824f84de-51ea-4435-9aee-9fbcde961d41" volumeName="kubernetes.io/empty-dir/824f84de-51ea-4435-9aee-9fbcde961d41-tmp" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389876    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a" volumeName="kubernetes.io/configmap/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-cni-binary-copy" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389892    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1735618-e0f6-49b1-ade6-8fef300791a1" volumeName="kubernetes.io/secret/b1735618-e0f6-49b1-ade6-8fef300791a1-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389907    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e3f102b3-d7bb-4ae4-b3cf-c20dd774804d" volumeName="kubernetes.io/projected/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-kube-api-access-vtpk7" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389923    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ae0bc76-971d-4020-972f-6fb355e71aa0" volumeName="kubernetes.io/secret/2ae0bc76-971d-4020-972f-6fb355e71aa0-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389935    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a9c43ecd-b447-4e07-95cf-01523d71dda9" volumeName="kubernetes.io/secret/a9c43ecd-b447-4e07-95cf-01523d71dda9-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389969    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1735618-e0f6-49b1-ade6-8fef300791a1" volumeName="kubernetes.io/configmap/b1735618-e0f6-49b1-ade6-8fef300791a1-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.389988    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf17e11d-1428-47c8-9aea-9f6b7c831eb1" volumeName="kubernetes.io/configmap/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390001    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf17e11d-1428-47c8-9aea-9f6b7c831eb1" volumeName="kubernetes.io/configmap/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-metrics-server-audit-profiles" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390017    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf17e11d-1428-47c8-9aea-9f6b7c831eb1" volumeName="kubernetes.io/empty-dir/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-audit-log" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390028    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf17e11d-1428-47c8-9aea-9f6b7c831eb1" volumeName="kubernetes.io/secret/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-client-ca-bundle" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390041    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4e65089-4bdd-431f-b338-1596fbb4b02b" volumeName="kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-config" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390063    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="220393aa-93ee-4113-b373-ecd27198f170" volumeName="kubernetes.io/configmap/220393aa-93ee-4113-b373-ecd27198f170-metrics-client-ca" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390076    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8876fbb5-cdb6-4755-9122-8640d5c86560" volumeName="kubernetes.io/configmap/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390095    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8876fbb5-cdb6-4755-9122-8640d5c86560" volumeName="kubernetes.io/secret/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390111    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1735618-e0f6-49b1-ade6-8fef300791a1" volumeName="kubernetes.io/configmap/b1735618-e0f6-49b1-ade6-8fef300791a1-env-overrides" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390128    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e4104cc2-a8a5-4985-83cd-e593bf12a68f" volumeName="kubernetes.io/configmap/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390145    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e4104cc2-a8a5-4985-83cd-e593bf12a68f" volumeName="kubernetes.io/projected/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-api-access-42nvt" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390158    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7ed38b7-0997-4cd8-9da0-41093596a6a0" volumeName="kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-web-config" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390175    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0c816c10-1c32-4538-8724-834ebb405960" volumeName="kubernetes.io/configmap/0c816c10-1c32-4538-8724-834ebb405960-iptables-alerter-script" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390187    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d22b0cdb-141d-4710-9f1f-90797d952a54" volumeName="kubernetes.io/configmap/d22b0cdb-141d-4710-9f1f-90797d952a54-config-volume" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390200    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d3fa116f-e372-4a36-9a57-4832dc1440b3" volumeName="kubernetes.io/empty-dir/d3fa116f-e372-4a36-9a57-4832dc1440b3-tmp-dir" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390217    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e3f102b3-d7bb-4ae4-b3cf-c20dd774804d" volumeName="kubernetes.io/empty-dir/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-tmp" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390229    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e3f102b3-d7bb-4ae4-b3cf-c20dd774804d" volumeName="kubernetes.io/empty-dir/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-tuned" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390246    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e4104cc2-a8a5-4985-83cd-e593bf12a68f" volumeName="kubernetes.io/configmap/e4104cc2-a8a5-4985-83cd-e593bf12a68f-metrics-client-ca" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390259    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7ed38b7-0997-4cd8-9da0-41093596a6a0" volumeName="kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-cluster-tls-config" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390290    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7ed38b7-0997-4cd8-9da0-41093596a6a0" volumeName="kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-main-tls" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390306    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="08a8e709-63af-4d71-b084-cf533ea620e5" volumeName="kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390318    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1538afd4-e5de-4372-aa9c-895efc60d643" volumeName="kubernetes.io/projected/1538afd4-e5de-4372-aa9c-895efc60d643-kube-api-access-sbhr8" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390334    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="220393aa-93ee-4113-b373-ecd27198f170" volumeName="kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390348    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ae0bc76-971d-4020-972f-6fb355e71aa0" volumeName="kubernetes.io/configmap/2ae0bc76-971d-4020-972f-6fb355e71aa0-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390360    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84" volumeName="kubernetes.io/secret/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-prometheus-operator-tls" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390379    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d22b0cdb-141d-4710-9f1f-90797d952a54" volumeName="kubernetes.io/secret/d22b0cdb-141d-4710-9f1f-90797d952a54-metrics-tls" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390392    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4e65089-4bdd-431f-b338-1596fbb4b02b" volumeName="kubernetes.io/secret/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-serving-cert" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390408    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e5c45faf-cafe-4ee2-9faa-f8686b13c4bd" volumeName="kubernetes.io/configmap/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-cni-binary-copy" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390421    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="08a8e709-63af-4d71-b084-cf533ea620e5" volumeName="kubernetes.io/projected/08a8e709-63af-4d71-b084-cf533ea620e5-kube-api-access-25k6c" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390434    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1538afd4-e5de-4372-aa9c-895efc60d643" volumeName="kubernetes.io/secret/1538afd4-e5de-4372-aa9c-895efc60d643-cert" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390452    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="824f84de-51ea-4435-9aee-9fbcde961d41" volumeName="kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-service-ca" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390463    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7ed38b7-0997-4cd8-9da0-41093596a6a0" volumeName="kubernetes.io/projected/f7ed38b7-0997-4cd8-9da0-41093596a6a0-tls-assets" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390479    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7ed38b7-0997-4cd8-9da0-41093596a6a0" volumeName="kubernetes.io/projected/f7ed38b7-0997-4cd8-9da0-41093596a6a0-kube-api-access-d4bcf" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390492    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7ed38b7-0997-4cd8-9da0-41093596a6a0" volumeName="kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390504    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="220393aa-93ee-4113-b373-ecd27198f170" volumeName="kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390522    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7bfc481b-9824-43c5-acf0-436641114b03" volumeName="kubernetes.io/projected/7bfc481b-9824-43c5-acf0-436641114b03-kube-api-access-5g5sn" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390535    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a9c43ecd-b447-4e07-95cf-01523d71dda9" volumeName="kubernetes.io/secret/a9c43ecd-b447-4e07-95cf-01523d71dda9-openshift-state-metrics-tls" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390551    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e5c45faf-cafe-4ee2-9faa-f8686b13c4bd" volumeName="kubernetes.io/configmap/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-daemon-config" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390563    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="220393aa-93ee-4113-b373-ecd27198f170" volumeName="kubernetes.io/projected/220393aa-93ee-4113-b373-ecd27198f170-kube-api-access-j7rqz" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390575    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="220393aa-93ee-4113-b373-ecd27198f170" volumeName="kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-tls" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390593    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ae0bc76-971d-4020-972f-6fb355e71aa0" volumeName="kubernetes.io/empty-dir/2ae0bc76-971d-4020-972f-6fb355e71aa0-data-volume" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390610    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8876fbb5-cdb6-4755-9122-8640d5c86560" volumeName="kubernetes.io/projected/8876fbb5-cdb6-4755-9122-8640d5c86560-kube-api-access-km4hl" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390628    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf17e11d-1428-47c8-9aea-9f6b7c831eb1" volumeName="kubernetes.io/secret/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-secret-metrics-server-tls" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390641    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4e65089-4bdd-431f-b338-1596fbb4b02b" volumeName="kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-oauth-serving-cert" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390653    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84" volumeName="kubernetes.io/secret/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390670    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="824f84de-51ea-4435-9aee-9fbcde961d41" volumeName="kubernetes.io/secret/824f84de-51ea-4435-9aee-9fbcde961d41-console-oauth-config" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390683    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7ed38b7-0997-4cd8-9da0-41093596a6a0" volumeName="kubernetes.io/empty-dir/f7ed38b7-0997-4cd8-9da0-41093596a6a0-config-out" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390702    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7bfc481b-9824-43c5-acf0-436641114b03" volumeName="kubernetes.io/configmap/7bfc481b-9824-43c5-acf0-436641114b03-serviceca" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390714    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a" volumeName="kubernetes.io/configmap/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390726    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a" volumeName="kubernetes.io/configmap/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390742    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a864943c-d7eb-4486-99a6-7961573b57d4" volumeName="kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390753    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4e65089-4bdd-431f-b338-1596fbb4b02b" volumeName="kubernetes.io/empty-dir/d4e65089-4bdd-431f-b338-1596fbb4b02b-tmp" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390770    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7ed38b7-0997-4cd8-9da0-41093596a6a0" volumeName="kubernetes.io/empty-dir/f7ed38b7-0997-4cd8-9da0-41093596a6a0-alertmanager-main-db" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390784    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="220393aa-93ee-4113-b373-ecd27198f170" volumeName="kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-grpc-tls" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390801    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="220393aa-93ee-4113-b373-ecd27198f170" volumeName="kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390814    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="824f84de-51ea-4435-9aee-9fbcde961d41" volumeName="kubernetes.io/secret/824f84de-51ea-4435-9aee-9fbcde961d41-console-serving-cert" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390827    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8876fbb5-cdb6-4755-9122-8640d5c86560" volumeName="kubernetes.io/empty-dir/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-textfile" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390845    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a9c43ecd-b447-4e07-95cf-01523d71dda9" volumeName="kubernetes.io/projected/a9c43ecd-b447-4e07-95cf-01523d71dda9-kube-api-access-l2r2b" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390858    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1735618-e0f6-49b1-ade6-8fef300791a1" volumeName="kubernetes.io/configmap/b1735618-e0f6-49b1-ade6-8fef300791a1-ovnkube-config" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390875    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7ed38b7-0997-4cd8-9da0-41093596a6a0" volumeName="kubernetes.io/configmap/f7ed38b7-0997-4cd8-9da0-41093596a6a0-alertmanager-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390889    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7ed38b7-0997-4cd8-9da0-41093596a6a0" volumeName="kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-config-volume" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390900    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3443d97d-774b-4df3-abe3-3ca2d6b574b6" volumeName="kubernetes.io/configmap/3443d97d-774b-4df3-abe3-3ca2d6b574b6-konnectivity-ca" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390919    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="824f84de-51ea-4435-9aee-9fbcde961d41" volumeName="kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-oauth-serving-cert" seLinuxMountContext=""
Oct 30 18:34:39.390829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.390931    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="824f84de-51ea-4435-9aee-9fbcde961d41" volumeName="kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 18:34:39.395929 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.391042    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8876fbb5-cdb6-4755-9122-8640d5c86560" volumeName="kubernetes.io/configmap/8876fbb5-cdb6-4755-9122-8640d5c86560-metrics-client-ca" seLinuxMountContext=""
Oct 30 18:34:39.395929 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.391065    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a9c43ecd-b447-4e07-95cf-01523d71dda9" volumeName="kubernetes.io/configmap/a9c43ecd-b447-4e07-95cf-01523d71dda9-metrics-client-ca" seLinuxMountContext=""
Oct 30 18:34:39.395929 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.391078    5820 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e4104cc2-a8a5-4985-83cd-e593bf12a68f" volumeName="kubernetes.io/secret/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-state-metrics-tls" seLinuxMountContext=""
Oct 30 18:34:39.395929 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.391090    5820 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 18:34:39.395929 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.391098    5820 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 18:34:39.395929 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.395571    5820 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:28:27 +0000 UTC" deadline="2028-03-26 12:52:12.919943263 +0000 UTC"
Oct 30 18:34:39.395929 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.395615    5820 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12306h17m33.52433198s"
Oct 30 18:34:39.427003 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.426905    5820 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 18:34:39.428709 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.428681    5820 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 18:34:39.428709 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.428704    5820 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 18:34:39.428871 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.428719    5820 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 18:34:39.428871 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:34:39.428752    5820 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 18:34:39.447605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.447555    5820 generic.go:356] "Generic (PLEG): container finished" podID="8876fbb5-cdb6-4755-9122-8640d5c86560" containerID="6ae1cf197f8b00632c725672f174a89a01ca2aaa0c558a30ecdee9beed00a8d0" exitCode=0
Oct 30 18:34:39.467493 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.467415    5820 generic.go:356] "Generic (PLEG): container finished" podID="a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a" containerID="520723f61c5004eac96e043770d58d711e55738cbc755afae571f2242dba868e" exitCode=0
Oct 30 18:34:39.467493 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.467443    5820 generic.go:356] "Generic (PLEG): container finished" podID="a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a" containerID="ed3d244e3f6496b096d0728644592305e87c7e17d7a3f234286596dbbcb6ddc6" exitCode=0
Oct 30 18:34:39.467493 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.467453    5820 generic.go:356] "Generic (PLEG): container finished" podID="a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a" containerID="85c487d7d0b52c1ea626aa5e7ca616c6be6973bbd5bedaf15dab859d20a8487a" exitCode=0
Oct 30 18:34:39.467493 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.467462    5820 generic.go:356] "Generic (PLEG): container finished" podID="a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a" containerID="1f450a1906406d89aa98aee291c7232d4ccbd52b478be1e75b599bac05f23d80" exitCode=0
Oct 30 18:34:39.467493 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.467470    5820 generic.go:356] "Generic (PLEG): container finished" podID="a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a" containerID="00bb5b7ca5c534e5f09448453fcfdd026c68fb2596b29e88b44cc0f90e10c164" exitCode=0
Oct 30 18:34:39.467493 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.467481    5820 generic.go:356] "Generic (PLEG): container finished" podID="a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a" containerID="10f25d7d2a0d5a40f6107a0c66f7e75a751838b29911e73eeb02fc255766ebe6" exitCode=0
Oct 30 18:34:39.479089 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:34:39.479064    5820 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-80.us-west-1.compute.internal\" not found"
Oct 30 18:34:39.490096 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.490067    5820 generic.go:356] "Generic (PLEG): container finished" podID="d085b0a4a8eed9a1bd72245b7d7a6e5e" containerID="9c94177951b44bc3c0bdddcede8570f6c7214960dc3045f8b404949e9dc01728" exitCode=0
Oct 30 18:34:39.496972 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.496928    5820 generic.go:356] "Generic (PLEG): container finished" podID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerID="62e3c9bf7e576b3b3d45f029b9b4d48e7d2bdf1e1c03a0103679146beaf27a5a" exitCode=0
Oct 30 18:34:39.528820 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:34:39.528795    5820 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 30 18:34:39.534140 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.534122    5820 manager.go:324] Recovery completed
Oct 30 18:34:39.555967 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.555934    5820 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:34:39.557724 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.557704    5820 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:34:39.557844 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.557737    5820 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:34:39.557844 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.557748    5820 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:34:39.560187 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.560169    5820 cpu_manager.go:226] "Starting" policy="none"
Oct 30 18:34:39.560187 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.560182    5820 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 18:34:39.560187 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.560186    5820 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:34:39.560401 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.560198    5820 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 18:34:39.560401 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.560320    5820 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 18:34:39.560401 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.560330    5820 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 18:34:39.560401 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.560346    5820 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 18:34:39.560401 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.560351    5820 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 18:34:39.560401 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.560356    5820 policy_none.go:50] "Start"
Oct 30 18:34:39.560401 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.560365    5820 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 18:34:39.560401 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.560381    5820 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 18:34:39.560637 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.560514    5820 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 18:34:39.560637 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.560522    5820 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 18:34:39.560637 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.560527    5820 policy_none.go:44] "Start"
Oct 30 18:34:39.561036 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.561019    5820 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:34:39.561109 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.561049    5820 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:34:39.561109 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.561060    5820 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:34:39.565174 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.565150    5820 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 18:34:39.567648 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.567634    5820 manager.go:341] "Starting Device Plugin manager"
Oct 30 18:34:39.567734 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:34:39.567657    5820 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 18:34:39.567734 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.567665    5820 server.go:87] "Starting device plugin registration server"
Oct 30 18:34:39.568000 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.567986    5820 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 18:34:39.568058 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.568003    5820 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 18:34:39.568120 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.568079    5820 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 18:34:39.568210 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.568173    5820 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 18:34:39.568321 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.568223    5820 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 18:34:39.568321 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.568229    5820 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 18:34:39.568440 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.568284    5820 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:34:39.568205565Z","UUID":"29da8347-d0d7-4d6b-a5f6-10075c42216c","Handler":null,"Name":"","Endpoint":""}
Oct 30 18:34:39.570134 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:34:39.570106    5820 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 18:34:39.570239 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:34:39.570177    5820 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-80.us-west-1.compute.internal\" not found"
Oct 30 18:34:39.570706 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.570689    5820 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 18:34:39.570706 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.570710    5820 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 18:34:39.572628 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.572601    5820 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:34:39.572708 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.572672    5820 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 18:34:39.573665 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.573638    5820 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:34:39.620963 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.620934    5820 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:34:39.669110 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.669082    5820 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:34:39.670179 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.670157    5820 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:34:39.670278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.670191    5820 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:34:39.670278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.670202    5820 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-80.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:34:39.670278 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.670253    5820 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:34:39.670767 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.670741    5820 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:34:39.729863 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.729767    5820 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal"]
Oct 30 18:34:39.730472 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.730440    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal" event={"ID":"d085b0a4a8eed9a1bd72245b7d7a6e5e","Type":"ContainerStarted","Data":"bc3cb9b6374a791c53326adc2b6972d9424f180e393edb538789b4338c326d51"}
Oct 30 18:34:39.730562 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.730479    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal" event={"ID":"d085b0a4a8eed9a1bd72245b7d7a6e5e","Type":"ContainerDied","Data":"9c94177951b44bc3c0bdddcede8570f6c7214960dc3045f8b404949e9dc01728"}
Oct 30 18:34:39.730562 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.730493    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal" event={"ID":"d085b0a4a8eed9a1bd72245b7d7a6e5e","Type":"ContainerStarted","Data":"20eb7acf3821a2acf3f28ee6fde8dd72950a5c8a215d3e8d6c9bf99861bb6720"}
Oct 30 18:34:39.730562 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.730529    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal" event={"ID":"ff8defe0c0f1527a0efd6540e2b0bdc3","Type":"ContainerStarted","Data":"d5bef21a62e16b6a29db646890371946d3b4a1582c3ea77a0de9aff45391802f"}
Oct 30 18:34:39.730562 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.730541    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal" event={"ID":"ff8defe0c0f1527a0efd6540e2b0bdc3","Type":"ContainerStarted","Data":"212e735c50505ff6ea0b59b17796bc0e130cfb49f55ca39ae1055e8ada6c7223"}
Oct 30 18:34:39.730744 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.730572    5820 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:34:39.730744 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.730573    5820 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:34:39.779764 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.779742    5820 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:34:39.779864 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.779798    5820 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:34:39.798124 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.798101    5820 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:34:39.798248 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.798125    5820 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:34:39.798248 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:34:39.798138    5820 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:34:39.798248 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:34:39.798167    5820 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:34:39.874365 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.874344    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d085b0a4a8eed9a1bd72245b7d7a6e5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal\" (UID: \"d085b0a4a8eed9a1bd72245b7d7a6e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:34:39.874454 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.874374    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d085b0a4a8eed9a1bd72245b7d7a6e5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal\" (UID: \"d085b0a4a8eed9a1bd72245b7d7a6e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:34:39.874454 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.874417    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ff8defe0c0f1527a0efd6540e2b0bdc3-config\") pod \"kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal\" (UID: \"ff8defe0c0f1527a0efd6540e2b0bdc3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:34:39.974909 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.974877    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d085b0a4a8eed9a1bd72245b7d7a6e5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal\" (UID: \"d085b0a4a8eed9a1bd72245b7d7a6e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:34:39.975076 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.974922    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d085b0a4a8eed9a1bd72245b7d7a6e5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal\" (UID: \"d085b0a4a8eed9a1bd72245b7d7a6e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:34:39.975076 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.974944    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d085b0a4a8eed9a1bd72245b7d7a6e5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal\" (UID: \"d085b0a4a8eed9a1bd72245b7d7a6e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:34:39.975076 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.974971    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ff8defe0c0f1527a0efd6540e2b0bdc3-config\") pod \"kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal\" (UID: \"ff8defe0c0f1527a0efd6540e2b0bdc3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:34:39.975076 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.975015    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ff8defe0c0f1527a0efd6540e2b0bdc3-config\") pod \"kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal\" (UID: \"ff8defe0c0f1527a0efd6540e2b0bdc3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:34:39.975076 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:39.975023    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d085b0a4a8eed9a1bd72245b7d7a6e5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal\" (UID: \"d085b0a4a8eed9a1bd72245b7d7a6e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:34:40.363040 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.363009    5820 apiserver.go:52] "Watching apiserver"
Oct 30 18:34:40.516563 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.516528    5820 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 18:34:40.517271 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.517246    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal","openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv","openshift-console/console-59c4fb9dcd-xzxdr","openshift-monitoring/alertmanager-main-0","openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b","openshift-multus/network-metrics-daemon-h5nm6","openshift-insights/insights-runtime-extractor-klcg2","openshift-dns/dns-default-jdtqn","openshift-multus/multus-tzdf5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal","openshift-network-operator/iptables-alerter-dvxpz","openshift-monitoring/node-exporter-x9dhg","kube-system/global-pull-secret-syncer-gfs79","openshift-ovn-kubernetes/ovnkube-node-vq8lg","openshift-ingress-canary/ingress-canary-pxlmt","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5","openshift-monitoring/metrics-server-7445978df6-sf44q","openshift-console/console-ff87dfcf9-5kf5h","openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd","openshift-network-diagnostics/network-check-target-5gvjh","openshift-cluster-node-tuning-operator/tuned-nnld2","openshift-dns/node-resolver-nr7xl","openshift-image-registry/node-ca-8kldf","openshift-multus/multus-additional-cni-plugins-8jpvz","kube-system/konnectivity-agent-j4w9c","openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr","openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j"]
Oct 30 18:34:40.520181 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.520148    5820 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:34:40.520795 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.520773    5820 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:34:40.550815 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.550782    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:40.570609 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.570578    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 18:34:40.570729 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.570647    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-727z7\""
Oct 30 18:34:40.570729 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.570682    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cgz7p\""
Oct 30 18:34:40.570907 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.570884    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 18:34:40.570982 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.570891    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 18:34:40.570982 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.570976    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 18:34:40.571075 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.570990    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 18:34:40.571075 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.571014    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 18:34:40.573729 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.573708    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 18:34:40.573836 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.573819    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 18:34:40.573892 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.573842    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 18:34:40.573934 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.573900    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wkpbb\""
Oct 30 18:34:40.573934 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.573903    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 18:34:40.573934 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.573925    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 18:34:40.574052 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.573936    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qb49q\""
Oct 30 18:34:40.574052 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.573977    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 18:34:40.574052 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.574019    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cr7s6\""
Oct 30 18:34:40.576034 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.576022    5820 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:34:40.576077 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:34:40.576058    5820 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:34:40.578924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.578908    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 18:34:40.579004 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.578925    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 18:34:40.579004 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.578983    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-b7gnv\""
Oct 30 18:34:40.579077 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.579036    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4bpx7\""
Oct 30 18:34:40.579077 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.579070    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 18:34:40.579136 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.579071    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 18:34:40.579136 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.579084    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 18:34:40.579136 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.579121    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 18:34:40.579136 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.579129    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 18:34:40.579258 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.579180    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:34:40.579507 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.579495    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 18:34:40.579538 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.579517    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nh9hp\""
Oct 30 18:34:40.579575 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.579542    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-g6mvb\""
Oct 30 18:34:40.579881 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.579869    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 18:34:40.580153 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.580137    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-8wmd6\""
Oct 30 18:34:40.580153 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.580153    5820 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:34:40.580295 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:34:40.580209    5820 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-80.us-west-1.compute.internal"
Oct 30 18:34:40.580499 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.580484    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 18:34:40.580614 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.580597    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:34:40.580839 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.580826    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5qqcs\""
Oct 30 18:34:40.580906 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.580895    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 18:34:40.582064 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.582049    5820 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 18:34:40.584439 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.584427    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jkxcr\""
Oct 30 18:34:40.584487 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.584441    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 18:34:40.584525 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.584513    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 18:34:40.584578 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.584530    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 18:34:40.619749 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.619691    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 18:34:40.620007 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.619994    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 18:34:40.620056 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.620032    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 18:34:40.620486 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.620470    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 18:34:40.620589 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.620575    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 18:34:40.620725 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.620713    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lmrrc\""
Oct 30 18:34:40.620779 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.620766    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 18:34:40.620962 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.620934    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 18:34:40.621143 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.621129    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-b5h66\""
Oct 30 18:34:40.621275 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.621262    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 18:34:40.621321 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.621281    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 18:34:40.621395 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.621383    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tc629\""
Oct 30 18:34:40.621436 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.621394    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:34:40.621585 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.621571    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 18:34:40.622050 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.622037    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 18:34:40.622551 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.622538    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 18:34:40.622715 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.622701    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 18:34:40.623411 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.623397    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 18:34:40.623524 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.623512    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 18:34:40.623806 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.623789    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 18:34:40.623862 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.623824    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:34:40.623862 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.623858    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 18:34:40.628543 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.628527    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 18:34:40.628930 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.628916    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 18:34:40.629022 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.629008    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 18:34:40.629102 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.629083    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 18:34:40.629177 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.629114    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 18:34:40.629177 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.629117    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 18:34:40.629177 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.629144    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 18:34:40.629177 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.629154    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 18:34:40.629359 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.629189    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 18:34:40.629359 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.629241    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 18:34:40.629359 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.629256    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 18:34:40.629359 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.629294    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-gcq7x\""
Oct 30 18:34:40.629359 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.629349    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 18:34:40.629523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.629383    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-gg67k\""
Oct 30 18:34:40.629523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.629403    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qxrcd\""
Oct 30 18:34:40.629523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.629453    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 18:34:40.629605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.629525    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 18:34:40.629605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.629546    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-sqz88\""
Oct 30 18:34:40.629605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.629567    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 18:34:40.629682 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.629643    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 18:34:40.629914 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.629897    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 18:34:40.629981 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.629939    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 18:34:40.630031 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.629981    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 18:34:40.630031 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.629990    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 18:34:40.630116 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.630057    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 18:34:40.630643 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.630622    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 18:34:40.630719 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.630627    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-rwjgn\""
Oct 30 18:34:40.630821 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.630805    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-453p4sl0bkn6n\""
Oct 30 18:34:40.630881 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.630823    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-gxn6m\""
Oct 30 18:34:40.631079 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.631059    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 18:34:40.631280 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.631264    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 18:34:40.631385 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.631368    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 18:34:40.631899 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.631757    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 18:34:40.632006 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.631943    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-nxqnw\""
Oct 30 18:34:40.632085 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.632067    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 18:34:40.632236 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.632215    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 18:34:40.632413 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.632355    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 18:34:40.632413 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.632407    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k76xp\""
Oct 30 18:34:40.632524 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.632473    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-8ln3so4ei8qsv\""
Oct 30 18:34:40.632665 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.632635    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-wd6sn\""
Oct 30 18:34:40.632725 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.632673    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 18:34:40.633938 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.633919    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 18:34:40.634151 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.634135    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 18:34:40.671478 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.671459    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 18:34:40.679209 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679185    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-var-lib-cni-multus\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.679285 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679211    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-etc-selinux\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:34:40.679285 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679233    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-config\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:40.679285 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679249    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm96j\" (UniqueName: \"kubernetes.io/projected/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-kube-api-access-cm96j\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.679285 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679265    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-web-config\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.679428 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679317    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7ed38b7-0997-4cd8-9da0-41093596a6a0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.679428 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679397    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc7462f0-a45f-487a-99b9-3e89f783cb01-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-cgw6j\" (UID: \"dc7462f0-a45f-487a-99b9-3e89f783cb01\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j"
Oct 30 18:34:40.679586 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679560    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-sys\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.679678 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679589    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-config\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:40.679678 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679603    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-system-cni-dir\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:34:40.679678 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679637    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-system-cni-dir\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.679678 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679663    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-run-multus-certs\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.679862 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679676    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-web-config\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.679862 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679709    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc7462f0-a45f-487a-99b9-3e89f783cb01-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-cgw6j\" (UID: \"dc7462f0-a45f-487a-99b9-3e89f783cb01\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j"
Oct 30 18:34:40.679862 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679730    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:40.679862 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679753    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7ed38b7-0997-4cd8-9da0-41093596a6a0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.679862 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679760    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ae0bc76-971d-4020-972f-6fb355e71aa0-data-volume\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:40.679862 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679821    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-sysctl-conf\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.679862 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679823    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ae0bc76-971d-4020-972f-6fb355e71aa0-data-volume\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:40.680212 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679869    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-cni-dir\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.680212 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679887    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d22b0cdb-141d-4710-9f1f-90797d952a54-config-volume\") pod \"dns-default-jdtqn\" (UID: \"d22b0cdb-141d-4710-9f1f-90797d952a54\") " pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:40.680212 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679902    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1538afd4-e5de-4372-aa9c-895efc60d643-cert\") pod \"ingress-canary-pxlmt\" (UID: \"1538afd4-e5de-4372-aa9c-895efc60d643\") " pod="openshift-ingress-canary/ingress-canary-pxlmt"
Oct 30 18:34:40.680212 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679913    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:40.680212 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.679985    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-bmxnr\" (UID: \"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"
Oct 30 18:34:40.680212 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680015    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-cni-netd\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.680212 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680040    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:40.680212 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680065    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-service-ca\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:40.680212 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680104    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d22b0cdb-141d-4710-9f1f-90797d952a54-config-volume\") pod \"dns-default-jdtqn\" (UID: \"d22b0cdb-141d-4710-9f1f-90797d952a54\") " pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:40.680212 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680113    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8876fbb5-cdb6-4755-9122-8640d5c86560-metrics-client-ca\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:40.680212 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680142    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbhr8\" (UniqueName: \"kubernetes.io/projected/1538afd4-e5de-4372-aa9c-895efc60d643-kube-api-access-sbhr8\") pod \"ingress-canary-pxlmt\" (UID: \"1538afd4-e5de-4372-aa9c-895efc60d643\") " pod="openshift-ingress-canary/ingress-canary-pxlmt"
Oct 30 18:34:40.680212 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680170    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a9c43ecd-b447-4e07-95cf-01523d71dda9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-g4h9b\" (UID: \"a9c43ecd-b447-4e07-95cf-01523d71dda9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"
Oct 30 18:34:40.680212 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680176    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1538afd4-e5de-4372-aa9c-895efc60d643-cert\") pod \"ingress-canary-pxlmt\" (UID: \"1538afd4-e5de-4372-aa9c-895efc60d643\") " pod="openshift-ingress-canary/ingress-canary-pxlmt"
Oct 30 18:34:40.680212 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680196    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-trusted-ca-bundle\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:40.680212 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680206    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-bmxnr\" (UID: \"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680222    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680249    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-systemd\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680250    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-service-ca\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680272    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8876fbb5-cdb6-4755-9122-8640d5c86560-metrics-client-ca\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680286    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-daemon-config\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680313    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-bmxnr\" (UID: \"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680328    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680340    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-systemd-units\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680366    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ae0bc76-971d-4020-972f-6fb355e71aa0-crio-socket\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680431    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-serving-cert\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680462    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a9c43ecd-b447-4e07-95cf-01523d71dda9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-g4h9b\" (UID: \"a9c43ecd-b447-4e07-95cf-01523d71dda9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680471    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-daemon-config\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680506    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-bmxnr\" (UID: \"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680513    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2r2b\" (UniqueName: \"kubernetes.io/projected/a9c43ecd-b447-4e07-95cf-01523d71dda9-kube-api-access-l2r2b\") pod \"openshift-state-metrics-7bd7489957-g4h9b\" (UID: \"a9c43ecd-b447-4e07-95cf-01523d71dda9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680547    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f7ed38b7-0997-4cd8-9da0-41093596a6a0-config-out\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680545    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-trusted-ca-bundle\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680565    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680590    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-etc-kubernetes\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680617    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-cnibin\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680623    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f7ed38b7-0997-4cd8-9da0-41093596a6a0-config-out\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680634    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-serving-cert\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680638    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-run\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680661    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-cnibin\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680697    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680697    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b1735618-e0f6-49b1-ade6-8fef300791a1-env-overrides\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680737    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25k6c\" (UniqueName: \"kubernetes.io/projected/08a8e709-63af-4d71-b084-cf533ea620e5-kube-api-access-25k6c\") pod \"network-metrics-daemon-h5nm6\" (UID: \"08a8e709-63af-4d71-b084-cf533ea620e5\") " pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680797    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d3fa116f-e372-4a36-9a57-4832dc1440b3-hosts-file\") pod \"node-resolver-nr7xl\" (UID: \"d3fa116f-e372-4a36-9a57-4832dc1440b3\") " pod="openshift-dns/node-resolver-nr7xl"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680870    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-socket-dir-parent\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680887    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b1735618-e0f6-49b1-ade6-8fef300791a1-env-overrides\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680902    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e4104cc2-a8a5-4985-83cd-e593bf12a68f-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:40.680928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680935    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680984    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.680998    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e4104cc2-a8a5-4985-83cd-e593bf12a68f-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681054    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-sys-fs\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681079    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-client-ca-bundle\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681097    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-slash\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681116    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-run-netns\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681138    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681217    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-run-ovn\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681226    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681242    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b1735618-e0f6-49b1-ade6-8fef300791a1-ovn-node-metrics-cert\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681258    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-oauth-serving-cert\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681267    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-client-ca-bundle\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681289    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-wtmp\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681389    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-oauth-serving-cert\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681386    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681427    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-run-netns\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681445    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-run-ovn-kubernetes\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681461    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7bfc481b-9824-43c5-acf0-436641114b03-serviceca\") pod \"node-ca-8kldf\" (UID: \"7bfc481b-9824-43c5-acf0-436641114b03\") " pod="openshift-image-registry/node-ca-8kldf"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681476    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7ed38b7-0997-4cd8-9da0-41093596a6a0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681503    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b1735618-e0f6-49b1-ade6-8fef300791a1-ovn-node-metrics-cert\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681541    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/220393aa-93ee-4113-b373-ecd27198f170-tmp\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681581    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-grpc-tls\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681585    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681623    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/824f84de-51ea-4435-9aee-9fbcde961d41-console-oauth-config\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681647    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-console-config\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681645    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7ed38b7-0997-4cd8-9da0-41093596a6a0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681649    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7bfc481b-9824-43c5-acf0-436641114b03-serviceca\") pod \"node-ca-8kldf\" (UID: \"7bfc481b-9824-43c5-acf0-436641114b03\") " pod="openshift-image-registry/node-ca-8kldf"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681659    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/220393aa-93ee-4113-b373-ecd27198f170-tmp\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681757    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-cni-binary-copy\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681771    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/824f84de-51ea-4435-9aee-9fbcde961d41-console-oauth-config\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681782    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d22b0cdb-141d-4710-9f1f-90797d952a54-metrics-tls\") pod \"dns-default-jdtqn\" (UID: \"d22b0cdb-141d-4710-9f1f-90797d952a54\") " pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681796    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-grpc-tls\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681802    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-console-config\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681809    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-var-lib-openvswitch\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681853    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-run-openvswitch\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681886    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4104cc2-a8a5-4985-83cd-e593bf12a68f-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681892    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-cni-binary-copy\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681913    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-var-lib-kubelet\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.681971    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d22b0cdb-141d-4710-9f1f-90797d952a54-metrics-tls\") pod \"dns-default-jdtqn\" (UID: \"d22b0cdb-141d-4710-9f1f-90797d952a54\") " pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682057    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6300215e-2302-47ed-bf47-05a165277c08-kubelet-config\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682077    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f7ed38b7-0997-4cd8-9da0-41093596a6a0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682095    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682099    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4104cc2-a8a5-4985-83cd-e593bf12a68f-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682117    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px2j7\" (UniqueName: \"kubernetes.io/projected/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-kube-api-access-px2j7\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682137    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f7ed38b7-0997-4cd8-9da0-41093596a6a0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682133    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b1735618-e0f6-49b1-ade6-8fef300791a1-ovnkube-script-lib\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682214    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f7ed38b7-0997-4cd8-9da0-41093596a6a0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682232    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-tls\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682248    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4z99\" (UniqueName: \"kubernetes.io/projected/824f84de-51ea-4435-9aee-9fbcde961d41-kube-api-access-t4z99\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682278    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-socket-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682295    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682302    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-service-ca\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682327    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b1735618-e0f6-49b1-ade6-8fef300791a1-ovnkube-script-lib\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682341    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-hostroot\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682365    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f7ed38b7-0997-4cd8-9da0-41093596a6a0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.682326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682365    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8876fbb5-cdb6-4755-9122-8640d5c86560-sys\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682408    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-log-socket\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682438    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9c43ecd-b447-4e07-95cf-01523d71dda9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-g4h9b\" (UID: \"a9c43ecd-b447-4e07-95cf-01523d71dda9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682471    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3443d97d-774b-4df3-abe3-3ca2d6b574b6-konnectivity-ca\") pod \"konnectivity-agent-j4w9c\" (UID: \"3443d97d-774b-4df3-abe3-3ca2d6b574b6\") " pod="kube-system/konnectivity-agent-j4w9c"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682478    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-service-ca\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682493    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-tls\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682516    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-cni-binary-copy\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682553    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682593    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj846\" (UniqueName: \"kubernetes.io/projected/dc7462f0-a45f-487a-99b9-3e89f783cb01-kube-api-access-vj846\") pod \"monitoring-plugin-77f4f76594-cgw6j\" (UID: \"dc7462f0-a45f-487a-99b9-3e89f783cb01\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682644    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3443d97d-774b-4df3-abe3-3ca2d6b574b6-konnectivity-ca\") pod \"konnectivity-agent-j4w9c\" (UID: \"3443d97d-774b-4df3-abe3-3ca2d6b574b6\") " pod="kube-system/konnectivity-agent-j4w9c"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682678    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9c43ecd-b447-4e07-95cf-01523d71dda9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-g4h9b\" (UID: \"a9c43ecd-b447-4e07-95cf-01523d71dda9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682689    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c816c10-1c32-4538-8724-834ebb405960-host-slash\") pod \"iptables-alerter-dvxpz\" (UID: \"0c816c10-1c32-4538-8724-834ebb405960\") " pod="openshift-network-operator/iptables-alerter-dvxpz"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682698    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-cni-binary-copy\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682726    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/220393aa-93ee-4113-b373-ecd27198f170-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682776    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7bfc481b-9824-43c5-acf0-436641114b03-host\") pod \"node-ca-8kldf\" (UID: \"7bfc481b-9824-43c5-acf0-436641114b03\") " pod="openshift-image-registry/node-ca-8kldf"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682807    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99psk\" (UniqueName: \"kubernetes.io/projected/d3fa116f-e372-4a36-9a57-4832dc1440b3-kube-api-access-99psk\") pod \"node-resolver-nr7xl\" (UID: \"d3fa116f-e372-4a36-9a57-4832dc1440b3\") " pod="openshift-dns/node-resolver-nr7xl"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682845    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ae0bc76-971d-4020-972f-6fb355e71aa0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682919    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs\") pod \"network-metrics-daemon-h5nm6\" (UID: \"08a8e709-63af-4d71-b084-cf533ea620e5\") " pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.682982    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-lib-modules\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683002    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d22b0cdb-141d-4710-9f1f-90797d952a54-tmp-dir\") pod \"dns-default-jdtqn\" (UID: \"d22b0cdb-141d-4710-9f1f-90797d952a54\") " pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683017    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-kubelet\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683033    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rlnr\" (UniqueName: \"kubernetes.io/projected/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-kube-api-access-4rlnr\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683047    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-modprobe-d\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683064    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ae0bc76-971d-4020-972f-6fb355e71aa0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683110    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683121    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d22b0cdb-141d-4710-9f1f-90797d952a54-tmp-dir\") pod \"dns-default-jdtqn\" (UID: \"d22b0cdb-141d-4710-9f1f-90797d952a54\") " pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683141    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683130    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08a8e709-63af-4d71-b084-cf533ea620e5-metrics-certs\") pod \"network-metrics-daemon-h5nm6\" (UID: \"08a8e709-63af-4d71-b084-cf533ea620e5\") " pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683180    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/824f84de-51ea-4435-9aee-9fbcde961d41-console-serving-cert\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683282    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ae0bc76-971d-4020-972f-6fb355e71aa0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683306    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-tls\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683323    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-accelerators-collector-config\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683339    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-bmxnr\" (UID: \"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683368    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6300215e-2302-47ed-bf47-05a165277c08-dbus\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683396    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/824f84de-51ea-4435-9aee-9fbcde961d41-console-serving-cert\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683417    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ae0bc76-971d-4020-972f-6fb355e71aa0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683425    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-cni-bin\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683494    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6300215e-2302-47ed-bf47-05a165277c08-original-pull-secret\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683509    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5djbl\" (UniqueName: \"kubernetes.io/projected/2ae0bc76-971d-4020-972f-6fb355e71aa0-kube-api-access-5djbl\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683542    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-secret-metrics-server-client-certs\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683556    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-bmxnr\" (UID: \"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683560    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-tls\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683560    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-accelerators-collector-config\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683572    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683605    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-audit-log\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683621    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-sysconfig\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683643    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-sysctl-d\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683686    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-audit-log\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683687    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km4hl\" (UniqueName: \"kubernetes.io/projected/8876fbb5-cdb6-4755-9122-8640d5c86560-kube-api-access-km4hl\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683713    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683750    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-secret-metrics-server-client-certs\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683790    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zzhj\" (UniqueName: \"kubernetes.io/projected/b1735618-e0f6-49b1-ade6-8fef300791a1-kube-api-access-2zzhj\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683819    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683847    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-config-volume\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683866    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/220393aa-93ee-4113-b373-ecd27198f170-metrics-client-ca\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683925    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3443d97d-774b-4df3-abe3-3ca2d6b574b6-agent-certs\") pod \"konnectivity-agent-j4w9c\" (UID: \"3443d97d-774b-4df3-abe3-3ca2d6b574b6\") " pod="kube-system/konnectivity-agent-j4w9c"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683966    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-tuned\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.683992    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-textfile\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684014    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684018    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-run-systemd\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684047    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-tuned\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684055    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42nvt\" (UniqueName: \"kubernetes.io/projected/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-api-access-42nvt\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684097    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl92s\" (UniqueName: \"kubernetes.io/projected/0c816c10-1c32-4538-8724-834ebb405960-kube-api-access-zl92s\") pod \"iptables-alerter-dvxpz\" (UID: \"0c816c10-1c32-4538-8724-834ebb405960\") " pod="openshift-network-operator/iptables-alerter-dvxpz"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684129    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-textfile\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684159    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/220393aa-93ee-4113-b373-ecd27198f170-metrics-client-ca\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684183    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-config-volume\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684190    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g5sn\" (UniqueName: \"kubernetes.io/projected/7bfc481b-9824-43c5-acf0-436641114b03-kube-api-access-5g5sn\") pod \"node-ca-8kldf\" (UID: \"7bfc481b-9824-43c5-acf0-436641114b03\") " pod="openshift-image-registry/node-ca-8kldf"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684206    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3443d97d-774b-4df3-abe3-3ca2d6b574b6-agent-certs\") pod \"konnectivity-agent-j4w9c\" (UID: \"3443d97d-774b-4df3-abe3-3ca2d6b574b6\") " pod="kube-system/konnectivity-agent-j4w9c"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684273    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srphr\" (UniqueName: \"kubernetes.io/projected/d22b0cdb-141d-4710-9f1f-90797d952a54-kube-api-access-srphr\") pod \"dns-default-jdtqn\" (UID: \"d22b0cdb-141d-4710-9f1f-90797d952a54\") " pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684305    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-node-log\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684338    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.684368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684364    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684454    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-kubernetes\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684488    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-conf-dir\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684508    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9c43ecd-b447-4e07-95cf-01523d71dda9-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-g4h9b\" (UID: \"a9c43ecd-b447-4e07-95cf-01523d71dda9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684528    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684538    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684599    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-oauth-config\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684624    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlwnx\" (UniqueName: \"kubernetes.io/projected/d4e65089-4bdd-431f-b338-1596fbb4b02b-kube-api-access-tlwnx\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684694    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9c43ecd-b447-4e07-95cf-01523d71dda9-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-g4h9b\" (UID: \"a9c43ecd-b447-4e07-95cf-01523d71dda9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684727    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8876fbb5-cdb6-4755-9122-8640d5c86560-root\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684763    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-oauth-config\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684763    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684828    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d8jv\" (UniqueName: \"kubernetes.io/projected/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-kube-api-access-7d8jv\") pod \"prometheus-operator-9cd89b74c-bmxnr\" (UID: \"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684851    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdl78\" (UniqueName: \"kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78\") pod \"network-check-target-5gvjh\" (UID: \"a864943c-d7eb-4486-99a6-7961573b57d4\") " pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684865    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-host\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684889    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3fa116f-e372-4a36-9a57-4832dc1440b3-tmp-dir\") pod \"node-resolver-nr7xl\" (UID: \"d3fa116f-e372-4a36-9a57-4832dc1440b3\") " pod="openshift-dns/node-resolver-nr7xl"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684907    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-run-k8s-cni-cncf-io\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.684967    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-etc-openvswitch\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685005    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3fa116f-e372-4a36-9a57-4832dc1440b3-tmp-dir\") pod \"node-resolver-nr7xl\" (UID: \"d3fa116f-e372-4a36-9a57-4832dc1440b3\") " pod="openshift-dns/node-resolver-nr7xl"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685024    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4bcf\" (UniqueName: \"kubernetes.io/projected/f7ed38b7-0997-4cd8-9da0-41093596a6a0-kube-api-access-d4bcf\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685053    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7rqz\" (UniqueName: \"kubernetes.io/projected/220393aa-93ee-4113-b373-ecd27198f170-kube-api-access-j7rqz\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685087    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4e65089-4bdd-431f-b338-1596fbb4b02b-tmp\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685108    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-secret-metrics-server-tls\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685127    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-var-lib-cni-bin\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685143    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685167    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4e65089-4bdd-431f-b338-1596fbb4b02b-tmp\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685167    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-registration-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685218    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0c816c10-1c32-4538-8724-834ebb405960-iptables-alerter-script\") pod \"iptables-alerter-dvxpz\" (UID: \"0c816c10-1c32-4538-8724-834ebb405960\") " pod="openshift-network-operator/iptables-alerter-dvxpz"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685260    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-tmp\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685340    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-secret-metrics-server-tls\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685374    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0c816c10-1c32-4538-8724-834ebb405960-iptables-alerter-script\") pod \"iptables-alerter-dvxpz\" (UID: \"0c816c10-1c32-4538-8724-834ebb405960\") " pod="openshift-network-operator/iptables-alerter-dvxpz"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685380    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685375    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-tmp\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685432    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-var-lib-kubelet\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685436    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685490    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b1735618-e0f6-49b1-ade6-8fef300791a1-ovnkube-config\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685515    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685532    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/824f84de-51ea-4435-9aee-9fbcde961d41-tmp\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685535    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685547    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-oauth-serving-cert\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685618    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/824f84de-51ea-4435-9aee-9fbcde961d41-tmp\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685640    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b1735618-e0f6-49b1-ade6-8fef300791a1-ovnkube-config\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685651    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-metrics-server-audit-profiles\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685668    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-oauth-serving-cert\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685680    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms9kr\" (UniqueName: \"kubernetes.io/projected/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-kube-api-access-ms9kr\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685685    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685706    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-device-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685736    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtpk7\" (UniqueName: \"kubernetes.io/projected/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-kube-api-access-vtpk7\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685761    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.685874    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-metrics-server-audit-profiles\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:40.686559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.686009    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:34:40.723504 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.723479    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:40.751142 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.751120    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlwnx\" (UniqueName: \"kubernetes.io/projected/d4e65089-4bdd-431f-b338-1596fbb4b02b-kube-api-access-tlwnx\") pod \"console-ff87dfcf9-5kf5h\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") " pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:40.752335 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.752305    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5djbl\" (UniqueName: \"kubernetes.io/projected/2ae0bc76-971d-4020-972f-6fb355e71aa0-kube-api-access-5djbl\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:40.752545 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.752524    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2r2b\" (UniqueName: \"kubernetes.io/projected/a9c43ecd-b447-4e07-95cf-01523d71dda9-kube-api-access-l2r2b\") pod \"openshift-state-metrics-7bd7489957-g4h9b\" (UID: \"a9c43ecd-b447-4e07-95cf-01523d71dda9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g4h9b"
Oct 30 18:34:40.752610 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.752524    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbhr8\" (UniqueName: \"kubernetes.io/projected/1538afd4-e5de-4372-aa9c-895efc60d643-kube-api-access-sbhr8\") pod \"ingress-canary-pxlmt\" (UID: \"1538afd4-e5de-4372-aa9c-895efc60d643\") " pod="openshift-ingress-canary/ingress-canary-pxlmt"
Oct 30 18:34:40.752610 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.752580    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7rqz\" (UniqueName: \"kubernetes.io/projected/220393aa-93ee-4113-b373-ecd27198f170-kube-api-access-j7rqz\") pod \"thanos-querier-6d4bc4b8d6-8k8jd\" (UID: \"220393aa-93ee-4113-b373-ecd27198f170\") " pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:40.754049 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.754033    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm96j\" (UniqueName: \"kubernetes.io/projected/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-kube-api-access-cm96j\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.755808 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.755789    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl92s\" (UniqueName: \"kubernetes.io/projected/0c816c10-1c32-4538-8724-834ebb405960-kube-api-access-zl92s\") pod \"iptables-alerter-dvxpz\" (UID: \"0c816c10-1c32-4538-8724-834ebb405960\") " pod="openshift-network-operator/iptables-alerter-dvxpz"
Oct 30 18:34:40.755973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.755928    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25k6c\" (UniqueName: \"kubernetes.io/projected/08a8e709-63af-4d71-b084-cf533ea620e5-kube-api-access-25k6c\") pod \"network-metrics-daemon-h5nm6\" (UID: \"08a8e709-63af-4d71-b084-cf533ea620e5\") " pod="openshift-multus/network-metrics-daemon-h5nm6"
Oct 30 18:34:40.756583 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.756562    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rlnr\" (UniqueName: \"kubernetes.io/projected/cf17e11d-1428-47c8-9aea-9f6b7c831eb1-kube-api-access-4rlnr\") pod \"metrics-server-7445978df6-sf44q\" (UID: \"cf17e11d-1428-47c8-9aea-9f6b7c831eb1\") " pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:40.757600 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.757578    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-px2j7\" (UniqueName: \"kubernetes.io/projected/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-kube-api-access-px2j7\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:34:40.757793 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.757777    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zzhj\" (UniqueName: \"kubernetes.io/projected/b1735618-e0f6-49b1-ade6-8fef300791a1-kube-api-access-2zzhj\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.757881 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.757865    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdl78\" (UniqueName: \"kubernetes.io/projected/a864943c-d7eb-4486-99a6-7961573b57d4-kube-api-access-zdl78\") pod \"network-check-target-5gvjh\" (UID: \"a864943c-d7eb-4486-99a6-7961573b57d4\") " pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:34:40.757926 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.757913    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99psk\" (UniqueName: \"kubernetes.io/projected/d3fa116f-e372-4a36-9a57-4832dc1440b3-kube-api-access-99psk\") pod \"node-resolver-nr7xl\" (UID: \"d3fa116f-e372-4a36-9a57-4832dc1440b3\") " pod="openshift-dns/node-resolver-nr7xl"
Oct 30 18:34:40.758085 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.758069    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42nvt\" (UniqueName: \"kubernetes.io/projected/e4104cc2-a8a5-4985-83cd-e593bf12a68f-kube-api-access-42nvt\") pod \"kube-state-metrics-5dc8d7d89b-4krbv\" (UID: \"e4104cc2-a8a5-4985-83cd-e593bf12a68f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-4krbv"
Oct 30 18:34:40.759188 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.759163    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms9kr\" (UniqueName: \"kubernetes.io/projected/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-kube-api-access-ms9kr\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:34:40.759621 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.759602    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4z99\" (UniqueName: \"kubernetes.io/projected/824f84de-51ea-4435-9aee-9fbcde961d41-kube-api-access-t4z99\") pod \"console-59c4fb9dcd-xzxdr\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") " pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:40.759803 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.759789    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtpk7\" (UniqueName: \"kubernetes.io/projected/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-kube-api-access-vtpk7\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.761172 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.761158    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srphr\" (UniqueName: \"kubernetes.io/projected/d22b0cdb-141d-4710-9f1f-90797d952a54-kube-api-access-srphr\") pod \"dns-default-jdtqn\" (UID: \"d22b0cdb-141d-4710-9f1f-90797d952a54\") " pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:40.761310 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.761296    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g5sn\" (UniqueName: \"kubernetes.io/projected/7bfc481b-9824-43c5-acf0-436641114b03-kube-api-access-5g5sn\") pod \"node-ca-8kldf\" (UID: \"7bfc481b-9824-43c5-acf0-436641114b03\") " pod="openshift-image-registry/node-ca-8kldf"
Oct 30 18:34:40.761667 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.761653    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-km4hl\" (UniqueName: \"kubernetes.io/projected/8876fbb5-cdb6-4755-9122-8640d5c86560-kube-api-access-km4hl\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:40.762642 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.762628    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj846\" (UniqueName: \"kubernetes.io/projected/dc7462f0-a45f-487a-99b9-3e89f783cb01-kube-api-access-vj846\") pod \"monitoring-plugin-77f4f76594-cgw6j\" (UID: \"dc7462f0-a45f-487a-99b9-3e89f783cb01\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j"
Oct 30 18:34:40.763172 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.763157    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d8jv\" (UniqueName: \"kubernetes.io/projected/3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84-kube-api-access-7d8jv\") pod \"prometheus-operator-9cd89b74c-bmxnr\" (UID: \"3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bmxnr"
Oct 30 18:34:40.768674 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.768660    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4bcf\" (UniqueName: \"kubernetes.io/projected/f7ed38b7-0997-4cd8-9da0-41093596a6a0-kube-api-access-d4bcf\") pod \"alertmanager-main-0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:34:40.786173 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786152    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-var-lib-openvswitch\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.786262 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786177    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-run-openvswitch\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.786262 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786218    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-run-openvswitch\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.786262 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786216    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-var-lib-openvswitch\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.786262 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786234    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-var-lib-kubelet\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.786262 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786254    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6300215e-2302-47ed-bf47-05a165277c08-kubelet-config\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:34:40.786485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786272    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-socket-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:34:40.786485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786273    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-var-lib-kubelet\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.786485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786286    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-hostroot\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.786485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786300    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8876fbb5-cdb6-4755-9122-8640d5c86560-sys\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:40.786485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786311    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-socket-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:34:40.786485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786314    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-log-socket\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.786485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786330    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6300215e-2302-47ed-bf47-05a165277c08-kubelet-config\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:34:40.786485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786339    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-log-socket\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.786485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786348    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c816c10-1c32-4538-8724-834ebb405960-host-slash\") pod \"iptables-alerter-dvxpz\" (UID: \"0c816c10-1c32-4538-8724-834ebb405960\") " pod="openshift-network-operator/iptables-alerter-dvxpz"
Oct 30 18:34:40.786485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786368    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c816c10-1c32-4538-8724-834ebb405960-host-slash\") pod \"iptables-alerter-dvxpz\" (UID: \"0c816c10-1c32-4538-8724-834ebb405960\") " pod="openshift-network-operator/iptables-alerter-dvxpz"
Oct 30 18:34:40.786485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786373    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7bfc481b-9824-43c5-acf0-436641114b03-host\") pod \"node-ca-8kldf\" (UID: \"7bfc481b-9824-43c5-acf0-436641114b03\") " pod="openshift-image-registry/node-ca-8kldf"
Oct 30 18:34:40.786485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786386    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8876fbb5-cdb6-4755-9122-8640d5c86560-sys\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:40.786485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786401    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7bfc481b-9824-43c5-acf0-436641114b03-host\") pod \"node-ca-8kldf\" (UID: \"7bfc481b-9824-43c5-acf0-436641114b03\") " pod="openshift-image-registry/node-ca-8kldf"
Oct 30 18:34:40.786485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786400    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-lib-modules\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.786485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786426    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-hostroot\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.786485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786448    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-kubelet\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.786485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786431    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-kubelet\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.786485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786479    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-modprobe-d\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.786485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786495    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786513    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6300215e-2302-47ed-bf47-05a165277c08-dbus\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786533    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-cni-bin\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786512    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-lib-modules\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786559    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-sysconfig\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786583    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6300215e-2302-47ed-bf47-05a165277c08-dbus\") pod \"global-pull-secret-syncer-gfs79\" (UID: \"6300215e-2302-47ed-bf47-05a165277c08\") " pod="kube-system/global-pull-secret-syncer-gfs79"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786599    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-cni-bin\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786609    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786622    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-sysconfig\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786591    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-modprobe-d\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786584    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-sysctl-d\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786666    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-run-systemd\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786692    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-node-log\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786695    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-sysctl-d\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786729    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-node-log\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786752    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786772    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-run-systemd\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786773    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-kubernetes\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786811    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-kubernetes\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786811    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786819    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-conf-dir\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786866    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8876fbb5-cdb6-4755-9122-8640d5c86560-root\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786891    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-host\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786927    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-run-k8s-cni-cncf-io\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.786977    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-etc-openvswitch\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787002    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-var-lib-cni-bin\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787025    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-registration-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787064    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-var-lib-kubelet\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787093    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-device-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787134    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-var-lib-cni-multus\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787149    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-etc-openvswitch\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787163    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8876fbb5-cdb6-4755-9122-8640d5c86560-root\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787168    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-host\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787157    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-etc-selinux\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787451    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-sys\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787470    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-system-cni-dir\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787226    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-device-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787233    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-registration-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787531    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-system-cni-dir\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787269    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-var-lib-cni-bin\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787254    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-var-lib-kubelet\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787266    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-var-lib-cni-multus\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787548    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-system-cni-dir\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787491    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-system-cni-dir\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787201    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-conf-dir\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787291    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-run-k8s-cni-cncf-io\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787566    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-sys\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787252    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-etc-selinux\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787586    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-run-multus-certs\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787619    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-sysctl-conf\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.787627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787647    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-cni-dir\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787682    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-run-multus-certs\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787696    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-cni-dir\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787707    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-cni-netd\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787719    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-sysctl-conf\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787725    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787752    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787752    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-systemd\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787761    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-cni-netd\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787786    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-systemd-units\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787789    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-etc-systemd\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787816    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-systemd-units\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787834    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ae0bc76-971d-4020-972f-6fb355e71aa0-crio-socket\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787857    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-etc-kubernetes\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787873    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-cnibin\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787886    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-run\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787903    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-cnibin\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787887    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ae0bc76-971d-4020-972f-6fb355e71aa0-crio-socket\") pod \"insights-runtime-extractor-klcg2\" (UID: \"2ae0bc76-971d-4020-972f-6fb355e71aa0\") " pod="openshift-insights/insights-runtime-extractor-klcg2"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787905    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-etc-kubernetes\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787929    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a-cnibin\") pod \"multus-additional-cni-plugins-8jpvz\" (UID: \"a1cec39c-0df9-4b68-bf4c-8c58c65ebc7a\") " pod="openshift-multus/multus-additional-cni-plugins-8jpvz"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787929    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d3fa116f-e372-4a36-9a57-4832dc1440b3-hosts-file\") pod \"node-resolver-nr7xl\" (UID: \"d3fa116f-e372-4a36-9a57-4832dc1440b3\") " pod="openshift-dns/node-resolver-nr7xl"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787968    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e3f102b3-d7bb-4ae4-b3cf-c20dd774804d-run\") pod \"tuned-nnld2\" (UID: \"e3f102b3-d7bb-4ae4-b3cf-c20dd774804d\") " pod="openshift-cluster-node-tuning-operator/tuned-nnld2"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787994    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-socket-dir-parent\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.787998    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d3fa116f-e372-4a36-9a57-4832dc1440b3-hosts-file\") pod \"node-resolver-nr7xl\" (UID: \"d3fa116f-e372-4a36-9a57-4832dc1440b3\") " pod="openshift-dns/node-resolver-nr7xl"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.788010    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-cnibin\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.788020    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-sys-fs\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.788047    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-slash\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.788053    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-multus-socket-dir-parent\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.788066    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fb0fef6c-8c89-40af-bd03-99cb7035a3f2-sys-fs\") pod \"aws-ebs-csi-driver-node-m46t5\" (UID: \"fb0fef6c-8c89-40af-bd03-99cb7035a3f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m46t5"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.788069    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-run-netns\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.788090    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-slash\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.788097    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-run-ovn\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.788101    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-run-netns\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.788122    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-wtmp\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.788144    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-run-ovn\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.788153    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-run-netns\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.788169    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-run-ovn-kubernetes\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.788194    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8876fbb5-cdb6-4755-9122-8640d5c86560-node-exporter-wtmp\") pod \"node-exporter-x9dhg\" (UID: \"8876fbb5-cdb6-4755-9122-8640d5c86560\") " pod="openshift-monitoring/node-exporter-x9dhg"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.788197    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5c45faf-cafe-4ee2-9faa-f8686b13c4bd-host-run-netns\") pod \"multus-tzdf5\" (UID: \"e5c45faf-cafe-4ee2-9faa-f8686b13c4bd\") " pod="openshift-multus/multus-tzdf5"
Oct 30 18:34:40.789018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.788210    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1735618-e0f6-49b1-ade6-8fef300791a1-host-run-ovn-kubernetes\") pod \"ovnkube-node-vq8lg\" (UID: \"b1735618-e0f6-49b1-ade6-8fef300791a1\") " pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:40.919224 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:40.919156    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:41.377736 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:41.377698    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:41.603283 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:41.603248    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:42.138184 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:42.138151    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:34:42.358177 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:42.358141    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-ff87dfcf9-5kf5h"]
Oct 30 18:34:42.690414 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:42.690385    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:43.624028 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:43.624001    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jdtqn"
Oct 30 18:34:43.792355 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:43.792316    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j"
Oct 30 18:34:43.795889 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:43.795863    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cgw6j"
Oct 30 18:34:44.781837 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:44.781807    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:44.787425 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:44.787398    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6d4bc4b8d6-8k8jd"
Oct 30 18:34:44.809007 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:44.808983    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-76b7c98f9-cpqf9"]
Oct 30 18:34:44.813891 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:44.813870    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:44.914548 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:44.914517    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76b7c98f9-cpqf9"]
Oct 30 18:34:44.924917 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:44.924882    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37a53409-387e-4585-992f-ebbbc6c3b437-console-serving-cert\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:44.924917 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:44.924912    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8thzl\" (UniqueName: \"kubernetes.io/projected/37a53409-387e-4585-992f-ebbbc6c3b437-kube-api-access-8thzl\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:44.925135 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:44.925024    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-oauth-serving-cert\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:44.925135 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:44.925062    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-console-config\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:44.925135 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:44.925097    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-trusted-ca-bundle\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:44.925293 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:44.925162    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37a53409-387e-4585-992f-ebbbc6c3b437-tmp\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:44.925293 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:44.925182    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37a53409-387e-4585-992f-ebbbc6c3b437-console-oauth-config\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:44.925293 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:44.925206    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-service-ca\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:45.026168 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:45.026135    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37a53409-387e-4585-992f-ebbbc6c3b437-console-serving-cert\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:45.026168 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:45.026174    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8thzl\" (UniqueName: \"kubernetes.io/projected/37a53409-387e-4585-992f-ebbbc6c3b437-kube-api-access-8thzl\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:45.026373 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:45.026224    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-oauth-serving-cert\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:45.026373 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:45.026260    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-console-config\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:45.026489 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:45.026469    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-trusted-ca-bundle\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:45.026543 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:45.026517    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37a53409-387e-4585-992f-ebbbc6c3b437-tmp\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:45.026600 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:45.026553    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37a53409-387e-4585-992f-ebbbc6c3b437-console-oauth-config\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:45.026645 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:45.026571    5820 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 18:34:45.026645 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:45.026600    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-service-ca\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:45.027131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:45.027057    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37a53409-387e-4585-992f-ebbbc6c3b437-tmp\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:45.027131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:45.027097    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-console-config\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:45.027131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:45.027119    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-oauth-serving-cert\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:45.027544 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:45.027521    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-trusted-ca-bundle\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:45.027605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:45.027552    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-service-ca\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:45.029056 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:45.029038    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37a53409-387e-4585-992f-ebbbc6c3b437-console-oauth-config\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:45.029056 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:45.029045    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37a53409-387e-4585-992f-ebbbc6c3b437-console-serving-cert\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:45.087796 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:45.087766    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8thzl\" (UniqueName: \"kubernetes.io/projected/37a53409-387e-4585-992f-ebbbc6c3b437-kube-api-access-8thzl\") pod \"console-76b7c98f9-cpqf9\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") " pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:45.125345 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:45.125307    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:45.172993 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:45.172674    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:45.172993 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:45.172996    5820 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 18:34:45.501504 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:45.501432    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76b7c98f9-cpqf9"]
Oct 30 18:34:45.504771 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:34:45.504742    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37a53409_387e_4585_992f_ebbbc6c3b437.slice/crio-c1bbf97788089409bbf24b2d62c1f1139cd5cac3bedbf9440ccb221dc34d8434 WatchSource:0}: Error finding container c1bbf97788089409bbf24b2d62c1f1139cd5cac3bedbf9440ccb221dc34d8434: Status 404 returned error can't find the container with id c1bbf97788089409bbf24b2d62c1f1139cd5cac3bedbf9440ccb221dc34d8434
Oct 30 18:34:45.547581 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:45.547550    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76b7c98f9-cpqf9" event={"ID":"37a53409-387e-4585-992f-ebbbc6c3b437","Type":"ContainerStarted","Data":"c1bbf97788089409bbf24b2d62c1f1139cd5cac3bedbf9440ccb221dc34d8434"}
Oct 30 18:34:46.102867 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:46.102828    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:46.351478 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:46.351445    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:46.552846 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:46.552812    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76b7c98f9-cpqf9" event={"ID":"37a53409-387e-4585-992f-ebbbc6c3b437","Type":"ContainerStarted","Data":"07c1e2cb24e7ab8c9774cca17b1adb8e562d6e3d2886ddae7acec7dbe84fa36b"}
Oct 30 18:34:46.720546 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:46.719691    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-76b7c98f9-cpqf9" podStartSLOduration=2.719673991 podStartE2EDuration="2.719673991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:34:46.670086642 +0000 UTC m=+7.381835338" watchObservedRunningTime="2026-10-30 18:34:46.719673991 +0000 UTC m=+7.431422688"
Oct 30 18:34:46.982176 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:46.982113    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vq8lg"
Oct 30 18:34:46.991220 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:46.991192    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:47.189243 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:47.189199    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:48.024003 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:48.023967    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:48.117679 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:48.117640    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:48.298461 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:48.298432    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:48.835698 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:48.835672    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:34:49.103508 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:49.103435    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:49.299393 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:49.299362    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:49.813055 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:49.813027    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:49.844817 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:49.844795    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:34:51.398025 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:51.397988    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:51.402716 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:51.402695    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:51.575110 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:51.575084    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:34:51.724692 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:51.724609    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59c4fb9dcd-xzxdr"]
Oct 30 18:34:52.310690 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:52.310659    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:52.485690 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:52.485655    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:54.120676 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:54.120644    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:54.295138 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:54.295108    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:54.563662 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:54.563638    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-j4w9c"
Oct 30 18:34:56.077532 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:56.077496    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:57.345963 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:57.345927    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:34:58.729226 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:58.729192    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:58.950690 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:34:58.950662    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:35:00.032849 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:00.032806    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:35:00.205390 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:00.205352    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:35:07.378706 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.378659    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-ff87dfcf9-5kf5h" podUID="d4e65089-4bdd-431f-b338-1596fbb4b02b" containerName="console" containerID="cri-o://a353b018084a1b0711d5cd1443b435040145652aab2e4f4cb5ce7a1e6ceabe9c" gracePeriod=15
Oct 30 18:35:07.629407 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.629325    5820 generic.go:356] "Generic (PLEG): container finished" podID="d4e65089-4bdd-431f-b338-1596fbb4b02b" containerID="a353b018084a1b0711d5cd1443b435040145652aab2e4f4cb5ce7a1e6ceabe9c" exitCode=0
Oct 30 18:35:07.629546 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.629393    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ff87dfcf9-5kf5h" event={"ID":"d4e65089-4bdd-431f-b338-1596fbb4b02b","Type":"ContainerDied","Data":"a353b018084a1b0711d5cd1443b435040145652aab2e4f4cb5ce7a1e6ceabe9c"}
Oct 30 18:35:07.667645 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.667616    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:35:07.830173 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.830135    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-oauth-serving-cert\") pod \"d4e65089-4bdd-431f-b338-1596fbb4b02b\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") "
Oct 30 18:35:07.830346 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.830182    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4e65089-4bdd-431f-b338-1596fbb4b02b-tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4e65089-4bdd-431f-b338-1596fbb4b02b-tmp\") pod \"d4e65089-4bdd-431f-b338-1596fbb4b02b\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") "
Oct 30 18:35:07.830346 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.830215    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-service-ca\") pod \"d4e65089-4bdd-431f-b338-1596fbb4b02b\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") "
Oct 30 18:35:07.830346 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.830276    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-serving-cert\") pod \"d4e65089-4bdd-431f-b338-1596fbb4b02b\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") "
Oct 30 18:35:07.830346 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.830310    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-config\" (UniqueName: \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-config\") pod \"d4e65089-4bdd-431f-b338-1596fbb4b02b\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") "
Oct 30 18:35:07.830573 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.830354    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-oauth-config\") pod \"d4e65089-4bdd-431f-b338-1596fbb4b02b\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") "
Oct 30 18:35:07.830573 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.830470    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4e65089-4bdd-431f-b338-1596fbb4b02b-kube-api-access-tlwnx\" (UniqueName: \"kubernetes.io/projected/d4e65089-4bdd-431f-b338-1596fbb4b02b-kube-api-access-tlwnx\") pod \"d4e65089-4bdd-431f-b338-1596fbb4b02b\" (UID: \"d4e65089-4bdd-431f-b338-1596fbb4b02b\") "
Oct 30 18:35:07.830573 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.830501    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4e65089-4bdd-431f-b338-1596fbb4b02b-tmp" pod "d4e65089-4bdd-431f-b338-1596fbb4b02b" (UID: "d4e65089-4bdd-431f-b338-1596fbb4b02b"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:35:07.830727 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.830619    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-oauth-serving-cert" pod "d4e65089-4bdd-431f-b338-1596fbb4b02b" (UID: "d4e65089-4bdd-431f-b338-1596fbb4b02b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:35:07.830781 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.830759    5820 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-oauth-serving-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:07.830781 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.830776    5820 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4e65089-4bdd-431f-b338-1596fbb4b02b-tmp\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:07.831130 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.831097    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-service-ca" pod "d4e65089-4bdd-431f-b338-1596fbb4b02b" (UID: "d4e65089-4bdd-431f-b338-1596fbb4b02b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:35:07.831297 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.831168    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-config" pod "d4e65089-4bdd-431f-b338-1596fbb4b02b" (UID: "d4e65089-4bdd-431f-b338-1596fbb4b02b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:35:07.833340 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.833309    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-oauth-config" pod "d4e65089-4bdd-431f-b338-1596fbb4b02b" (UID: "d4e65089-4bdd-431f-b338-1596fbb4b02b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:35:07.833433 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.833352    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4e65089-4bdd-431f-b338-1596fbb4b02b-kube-api-access-tlwnx" pod "d4e65089-4bdd-431f-b338-1596fbb4b02b" (UID: "d4e65089-4bdd-431f-b338-1596fbb4b02b"). InnerVolumeSpecName "kube-api-access-tlwnx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:35:07.833433 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.833358    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-serving-cert" pod "d4e65089-4bdd-431f-b338-1596fbb4b02b" (UID: "d4e65089-4bdd-431f-b338-1596fbb4b02b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:35:07.931205 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.931127    5820 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-serving-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:07.931205 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.931156    5820 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-config\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:07.931205 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.931170    5820 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4e65089-4bdd-431f-b338-1596fbb4b02b-console-oauth-config\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:07.931205 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.931182    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlwnx\" (UniqueName: \"kubernetes.io/projected/d4e65089-4bdd-431f-b338-1596fbb4b02b-kube-api-access-tlwnx\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:07.931205 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:07.931196    5820 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4e65089-4bdd-431f-b338-1596fbb4b02b-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:08.202277 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:35:08.202196    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:35:08.634192 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:08.634156    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ff87dfcf9-5kf5h"
Oct 30 18:35:08.634640 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:08.634161    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ff87dfcf9-5kf5h" event={"ID":"d4e65089-4bdd-431f-b338-1596fbb4b02b","Type":"ContainerDied","Data":"43cd4cc8435d93fd0c9c87d3f00169ba3775f56ea3e2eafdadb56403d61fc493"}
Oct 30 18:35:08.634640 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:08.634322    5820 scope.go:122] "RemoveContainer" containerID="a353b018084a1b0711d5cd1443b435040145652aab2e4f4cb5ce7a1e6ceabe9c"
Oct 30 18:35:08.797516 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:08.797484    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-ff87dfcf9-5kf5h"]
Oct 30 18:35:08.846901 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:08.846872    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-ff87dfcf9-5kf5h"]
Oct 30 18:35:09.433826 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:09.433794    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d4e65089-4bdd-431f-b338-1596fbb4b02b" path="/var/lib/kubelet/pods/d4e65089-4bdd-431f-b338-1596fbb4b02b/volumes"
Oct 30 18:35:12.141047 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:12.141021    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5gvjh"
Oct 30 18:35:16.082046 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:16.082015    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:35:16.085750 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:16.085730    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7445978df6-sf44q"
Oct 30 18:35:16.752601 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:16.752548    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-59c4fb9dcd-xzxdr" podUID="824f84de-51ea-4435-9aee-9fbcde961d41" containerName="console" containerID="cri-o://d84ca3f135c80df8ac7a13e40589d5cd5b485bf1f3846846350988f1089020f4" gracePeriod=15
Oct 30 18:35:16.988815 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:16.988789    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:35:17.109268 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.109239    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/824f84de-51ea-4435-9aee-9fbcde961d41-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/824f84de-51ea-4435-9aee-9fbcde961d41-console-oauth-config\") pod \"824f84de-51ea-4435-9aee-9fbcde961d41\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") "
Oct 30 18:35:17.109683 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.109283    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-console-config\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-console-config\") pod \"824f84de-51ea-4435-9aee-9fbcde961d41\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") "
Oct 30 18:35:17.109683 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.109311    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/824f84de-51ea-4435-9aee-9fbcde961d41-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/824f84de-51ea-4435-9aee-9fbcde961d41-console-serving-cert\") pod \"824f84de-51ea-4435-9aee-9fbcde961d41\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") "
Oct 30 18:35:17.109683 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.109366    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-trusted-ca-bundle\") pod \"824f84de-51ea-4435-9aee-9fbcde961d41\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") "
Oct 30 18:35:17.109683 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.109395    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/824f84de-51ea-4435-9aee-9fbcde961d41-kube-api-access-t4z99\" (UniqueName: \"kubernetes.io/projected/824f84de-51ea-4435-9aee-9fbcde961d41-kube-api-access-t4z99\") pod \"824f84de-51ea-4435-9aee-9fbcde961d41\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") "
Oct 30 18:35:17.109683 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.109429    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-oauth-serving-cert\") pod \"824f84de-51ea-4435-9aee-9fbcde961d41\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") "
Oct 30 18:35:17.109683 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.109484    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/824f84de-51ea-4435-9aee-9fbcde961d41-tmp\" (UniqueName: \"kubernetes.io/empty-dir/824f84de-51ea-4435-9aee-9fbcde961d41-tmp\") pod \"824f84de-51ea-4435-9aee-9fbcde961d41\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") "
Oct 30 18:35:17.109683 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.109515    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-service-ca\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-service-ca\") pod \"824f84de-51ea-4435-9aee-9fbcde961d41\" (UID: \"824f84de-51ea-4435-9aee-9fbcde961d41\") "
Oct 30 18:35:17.110067 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.109761    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/824f84de-51ea-4435-9aee-9fbcde961d41-tmp" pod "824f84de-51ea-4435-9aee-9fbcde961d41" (UID: "824f84de-51ea-4435-9aee-9fbcde961d41"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:35:17.110067 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.109778    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-console-config" pod "824f84de-51ea-4435-9aee-9fbcde961d41" (UID: "824f84de-51ea-4435-9aee-9fbcde961d41"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:35:17.110067 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.109856    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-trusted-ca-bundle" pod "824f84de-51ea-4435-9aee-9fbcde961d41" (UID: "824f84de-51ea-4435-9aee-9fbcde961d41"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:35:17.110067 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.109875    5820 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/824f84de-51ea-4435-9aee-9fbcde961d41-tmp\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:17.110067 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.109967    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-oauth-serving-cert" pod "824f84de-51ea-4435-9aee-9fbcde961d41" (UID: "824f84de-51ea-4435-9aee-9fbcde961d41"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:35:17.110318 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.110258    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-service-ca" pod "824f84de-51ea-4435-9aee-9fbcde961d41" (UID: "824f84de-51ea-4435-9aee-9fbcde961d41"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:35:17.112777 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.112748    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/824f84de-51ea-4435-9aee-9fbcde961d41-console-oauth-config" pod "824f84de-51ea-4435-9aee-9fbcde961d41" (UID: "824f84de-51ea-4435-9aee-9fbcde961d41"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:35:17.112917 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.112877    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/824f84de-51ea-4435-9aee-9fbcde961d41-console-serving-cert" pod "824f84de-51ea-4435-9aee-9fbcde961d41" (UID: "824f84de-51ea-4435-9aee-9fbcde961d41"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:35:17.113715 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.113698    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/824f84de-51ea-4435-9aee-9fbcde961d41-kube-api-access-t4z99" pod "824f84de-51ea-4435-9aee-9fbcde961d41" (UID: "824f84de-51ea-4435-9aee-9fbcde961d41"). InnerVolumeSpecName "kube-api-access-t4z99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:35:17.211242 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.211211    5820 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-trusted-ca-bundle\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:17.211242 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.211236    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4z99\" (UniqueName: \"kubernetes.io/projected/824f84de-51ea-4435-9aee-9fbcde961d41-kube-api-access-t4z99\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:17.211242 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.211248    5820 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-oauth-serving-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:17.211457 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.211257    5820 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:17.211457 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.211267    5820 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/824f84de-51ea-4435-9aee-9fbcde961d41-console-oauth-config\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:17.211457 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.211275    5820 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/824f84de-51ea-4435-9aee-9fbcde961d41-console-config\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:17.211457 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.211284    5820 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/824f84de-51ea-4435-9aee-9fbcde961d41-console-serving-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:17.673508 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.673476    5820 generic.go:356] "Generic (PLEG): container finished" podID="824f84de-51ea-4435-9aee-9fbcde961d41" containerID="d84ca3f135c80df8ac7a13e40589d5cd5b485bf1f3846846350988f1089020f4" exitCode=0
Oct 30 18:35:17.673692 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.673527    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c4fb9dcd-xzxdr"
Oct 30 18:35:17.673692 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.673552    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c4fb9dcd-xzxdr" event={"ID":"824f84de-51ea-4435-9aee-9fbcde961d41","Type":"ContainerDied","Data":"d84ca3f135c80df8ac7a13e40589d5cd5b485bf1f3846846350988f1089020f4"}
Oct 30 18:35:17.673692 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.673601    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c4fb9dcd-xzxdr" event={"ID":"824f84de-51ea-4435-9aee-9fbcde961d41","Type":"ContainerDied","Data":"0515afe254d6c0481584c8483f6a489eca877016a7560058b6a645a864c7534a"}
Oct 30 18:35:17.673692 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.673623    5820 scope.go:122] "RemoveContainer" containerID="d84ca3f135c80df8ac7a13e40589d5cd5b485bf1f3846846350988f1089020f4"
Oct 30 18:35:17.681987 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.681970    5820 scope.go:122] "RemoveContainer" containerID="d84ca3f135c80df8ac7a13e40589d5cd5b485bf1f3846846350988f1089020f4"
Oct 30 18:35:17.682290 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:35:17.682266    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d84ca3f135c80df8ac7a13e40589d5cd5b485bf1f3846846350988f1089020f4\": container with ID starting with d84ca3f135c80df8ac7a13e40589d5cd5b485bf1f3846846350988f1089020f4 not found: ID does not exist" containerID="d84ca3f135c80df8ac7a13e40589d5cd5b485bf1f3846846350988f1089020f4"
Oct 30 18:35:17.682390 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.682293    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d84ca3f135c80df8ac7a13e40589d5cd5b485bf1f3846846350988f1089020f4"} err="failed to get container status \"d84ca3f135c80df8ac7a13e40589d5cd5b485bf1f3846846350988f1089020f4\": rpc error: code = NotFound desc = could not find container \"d84ca3f135c80df8ac7a13e40589d5cd5b485bf1f3846846350988f1089020f4\": container with ID starting with d84ca3f135c80df8ac7a13e40589d5cd5b485bf1f3846846350988f1089020f4 not found: ID does not exist"
Oct 30 18:35:17.836163 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.836128    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59c4fb9dcd-xzxdr"]
Oct 30 18:35:17.885828 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:17.885794    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59c4fb9dcd-xzxdr"]
Oct 30 18:35:18.127587 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:18.127492    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:35:18.308806 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:18.308771    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:35:19.434138 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:19.434105    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="824f84de-51ea-4435-9aee-9fbcde961d41" path="/var/lib/kubelet/pods/824f84de-51ea-4435-9aee-9fbcde961d41/volumes"
Oct 30 18:35:38.201774 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:35:38.201737    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:35:46.461960 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:46.461917    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:35:47.212780 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:47.212742    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:35:47.385914 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:47.385877    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:35:49.196573 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.196539    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f68b76cbb-tr2kf"]
Oct 30 18:35:49.196938 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.196822    5820 cpu_manager.go:414] "containerMap: removing container" podUID="824f84de-51ea-4435-9aee-9fbcde961d41" containerName="console"
Oct 30 18:35:49.196938 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.196833    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="824f84de-51ea-4435-9aee-9fbcde961d41" containerName="console"
Oct 30 18:35:49.196938 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.196856    5820 cpu_manager.go:414] "containerMap: removing container" podUID="d4e65089-4bdd-431f-b338-1596fbb4b02b" containerName="console"
Oct 30 18:35:49.196938 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.196861    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4e65089-4bdd-431f-b338-1596fbb4b02b" containerName="console"
Oct 30 18:35:49.196938 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.196917    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4e65089-4bdd-431f-b338-1596fbb4b02b" containerName="console"
Oct 30 18:35:49.196938 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.196927    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="824f84de-51ea-4435-9aee-9fbcde961d41" containerName="console"
Oct 30 18:35:49.198797 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.198767    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.243367 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.243343    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b5bc418-0485-49cb-b815-a42d6afc36bb-console-oauth-config\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.243510 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.243372    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b5bc418-0485-49cb-b815-a42d6afc36bb-service-ca\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.243510 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.243397    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6jrz\" (UniqueName: \"kubernetes.io/projected/3b5bc418-0485-49cb-b815-a42d6afc36bb-kube-api-access-n6jrz\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.243510 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.243484    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc418-0485-49cb-b815-a42d6afc36bb-tmp\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.243666 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.243522    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b5bc418-0485-49cb-b815-a42d6afc36bb-oauth-serving-cert\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.243666 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.243636    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b5bc418-0485-49cb-b815-a42d6afc36bb-console-serving-cert\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.243774 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.243674    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b5bc418-0485-49cb-b815-a42d6afc36bb-console-config\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.243774 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.243708    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b5bc418-0485-49cb-b815-a42d6afc36bb-trusted-ca-bundle\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.300941 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.300913    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f68b76cbb-tr2kf"]
Oct 30 18:35:49.345041 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.345014    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b5bc418-0485-49cb-b815-a42d6afc36bb-oauth-serving-cert\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.345182 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.345055    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b5bc418-0485-49cb-b815-a42d6afc36bb-console-serving-cert\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.345182 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.345077    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b5bc418-0485-49cb-b815-a42d6afc36bb-console-config\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.345182 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.345097    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b5bc418-0485-49cb-b815-a42d6afc36bb-trusted-ca-bundle\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.345326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.345220    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b5bc418-0485-49cb-b815-a42d6afc36bb-console-oauth-config\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.345326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.345266    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b5bc418-0485-49cb-b815-a42d6afc36bb-service-ca\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.345326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.345302    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6jrz\" (UniqueName: \"kubernetes.io/projected/3b5bc418-0485-49cb-b815-a42d6afc36bb-kube-api-access-n6jrz\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.345476 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.345332    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc418-0485-49cb-b815-a42d6afc36bb-tmp\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.345746 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.345725    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b5bc418-0485-49cb-b815-a42d6afc36bb-tmp\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.345911 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.345824    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b5bc418-0485-49cb-b815-a42d6afc36bb-oauth-serving-cert\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.346035 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.345845    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b5bc418-0485-49cb-b815-a42d6afc36bb-console-config\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.346084 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.346048    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b5bc418-0485-49cb-b815-a42d6afc36bb-trusted-ca-bundle\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.346084 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.346054    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b5bc418-0485-49cb-b815-a42d6afc36bb-service-ca\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.347846 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.347824    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b5bc418-0485-49cb-b815-a42d6afc36bb-console-oauth-config\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.347919 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.347909    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b5bc418-0485-49cb-b815-a42d6afc36bb-console-serving-cert\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.406153 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.406120    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6jrz\" (UniqueName: \"kubernetes.io/projected/3b5bc418-0485-49cb-b815-a42d6afc36bb-kube-api-access-n6jrz\") pod \"console-6f68b76cbb-tr2kf\" (UID: \"3b5bc418-0485-49cb-b815-a42d6afc36bb\") " pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.509032 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.508937    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:49.513275 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.513244    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:35:49.886884 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.886851    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f68b76cbb-tr2kf"]
Oct 30 18:35:49.887781 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:35:49.887758    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b5bc418_0485_49cb_b815_a42d6afc36bb.slice/crio-3d56b76490d8f7683cdd11a562bc5b5390f251d462400138aeb59c906e0dd6ab WatchSource:0}: Error finding container 3d56b76490d8f7683cdd11a562bc5b5390f251d462400138aeb59c906e0dd6ab: Status 404 returned error can't find the container with id 3d56b76490d8f7683cdd11a562bc5b5390f251d462400138aeb59c906e0dd6ab
Oct 30 18:35:49.909552 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:49.909522    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:35:50.085530 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:50.085501    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:35:50.797340 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:50.797303    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f68b76cbb-tr2kf" event={"ID":"3b5bc418-0485-49cb-b815-a42d6afc36bb","Type":"ContainerStarted","Data":"9b4d2bd420aecde7bcd8477b93c62ec97cf100cff4b0536aafd2ab9d64c74d1a"}
Oct 30 18:35:50.797716 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:50.797364    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f68b76cbb-tr2kf" event={"ID":"3b5bc418-0485-49cb-b815-a42d6afc36bb","Type":"ContainerStarted","Data":"3d56b76490d8f7683cdd11a562bc5b5390f251d462400138aeb59c906e0dd6ab"}
Oct 30 18:35:50.917867 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:50.917809    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6f68b76cbb-tr2kf" podStartSLOduration=1.9177942159999999 podStartE2EDuration="1.917794216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:35:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:35:50.917532374 +0000 UTC m=+71.629281074" watchObservedRunningTime="2026-10-30 18:35:50.917794216 +0000 UTC m=+71.629542912"
Oct 30 18:35:54.877777 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:54.877748    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 18:35:54.878963 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:54.878146    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="alertmanager" containerID="cri-o://7fdca55ec4308700260efd57276b0518d1eb3d50e1eada4def6468a83c846095" gracePeriod=120
Oct 30 18:35:54.878963 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:54.878846    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="kube-rbac-proxy" containerID="cri-o://ad5dc45c6ed151741af41b6243ad6704239b5ba43a4a62af9cff03c6b57983bd" gracePeriod=120
Oct 30 18:35:54.878963 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:54.878921    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="kube-rbac-proxy-web" containerID="cri-o://43a340d025c45535e367a6aea3e41cdb52b3bde8dd27e6f372180e2e436f5680" gracePeriod=120
Oct 30 18:35:54.879651 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:54.879007    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="config-reloader" containerID="cri-o://812c6cf749ef9fd286430ba652f60a717c618ee8d85654b467b446cd4fb6b8db" gracePeriod=120
Oct 30 18:35:54.879651 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:54.879007    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="kube-rbac-proxy-metric" containerID="cri-o://f7a1ef9cc808c0dc378f29e3ac1afd1b1833574b00dda97ae81661cc76c966c1" gracePeriod=120
Oct 30 18:35:54.879651 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:54.878874    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="prom-label-proxy" containerID="cri-o://ecb8816873ee3bc29892301cdf67a02f36a68e05b30f5ce97a93854c9c8e4812" gracePeriod=120
Oct 30 18:35:55.822314 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:55.822280    5820 generic.go:356] "Generic (PLEG): container finished" podID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerID="ecb8816873ee3bc29892301cdf67a02f36a68e05b30f5ce97a93854c9c8e4812" exitCode=0
Oct 30 18:35:55.822314 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:55.822305    5820 generic.go:356] "Generic (PLEG): container finished" podID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerID="ad5dc45c6ed151741af41b6243ad6704239b5ba43a4a62af9cff03c6b57983bd" exitCode=0
Oct 30 18:35:55.822314 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:55.822314    5820 generic.go:356] "Generic (PLEG): container finished" podID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerID="812c6cf749ef9fd286430ba652f60a717c618ee8d85654b467b446cd4fb6b8db" exitCode=0
Oct 30 18:35:55.822314 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:55.822319    5820 generic.go:356] "Generic (PLEG): container finished" podID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerID="7fdca55ec4308700260efd57276b0518d1eb3d50e1eada4def6468a83c846095" exitCode=0
Oct 30 18:35:55.822579 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:55.822335    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7ed38b7-0997-4cd8-9da0-41093596a6a0","Type":"ContainerDied","Data":"ecb8816873ee3bc29892301cdf67a02f36a68e05b30f5ce97a93854c9c8e4812"}
Oct 30 18:35:55.822579 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:55.822408    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7ed38b7-0997-4cd8-9da0-41093596a6a0","Type":"ContainerDied","Data":"ad5dc45c6ed151741af41b6243ad6704239b5ba43a4a62af9cff03c6b57983bd"}
Oct 30 18:35:55.822579 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:55.822432    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7ed38b7-0997-4cd8-9da0-41093596a6a0","Type":"ContainerDied","Data":"812c6cf749ef9fd286430ba652f60a717c618ee8d85654b467b446cd4fb6b8db"}
Oct 30 18:35:55.822579 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:55.822454    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7ed38b7-0997-4cd8-9da0-41093596a6a0","Type":"ContainerDied","Data":"7fdca55ec4308700260efd57276b0518d1eb3d50e1eada4def6468a83c846095"}
Oct 30 18:35:56.125384 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.125361    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:56.201459 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.201434    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f7ed38b7-0997-4cd8-9da0-41093596a6a0-kube-api-access-d4bcf\" (UniqueName: \"kubernetes.io/projected/f7ed38b7-0997-4cd8-9da0-41093596a6a0-kube-api-access-d4bcf\") pod \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") "
Oct 30 18:35:56.201627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.201471    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-web-config\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-web-config\") pod \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") "
Oct 30 18:35:56.201627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.201500    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-main-tls\") pod \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") "
Oct 30 18:35:56.201627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.201527    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7ed38b7-0997-4cd8-9da0-41093596a6a0-config-out\" (UniqueName: \"kubernetes.io/empty-dir/f7ed38b7-0997-4cd8-9da0-41093596a6a0-config-out\") pod \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") "
Oct 30 18:35:56.201627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.201564    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f7ed38b7-0997-4cd8-9da0-41093596a6a0-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7ed38b7-0997-4cd8-9da0-41093596a6a0-metrics-client-ca\") pod \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") "
Oct 30 18:35:56.201627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.201613    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy\") pod \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") "
Oct 30 18:35:56.201907 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.201647    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7ed38b7-0997-4cd8-9da0-41093596a6a0-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f7ed38b7-0997-4cd8-9da0-41093596a6a0-alertmanager-main-db\") pod \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") "
Oct 30 18:35:56.201907 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.201680    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-config-volume\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-config-volume\") pod \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") "
Oct 30 18:35:56.201907 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.201719    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f7ed38b7-0997-4cd8-9da0-41093596a6a0-tls-assets\" (UniqueName: \"kubernetes.io/projected/f7ed38b7-0997-4cd8-9da0-41093596a6a0-tls-assets\") pod \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") "
Oct 30 18:35:56.201907 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.201752    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-cluster-tls-config\") pod \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") "
Oct 30 18:35:56.201907 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.201793    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") "
Oct 30 18:35:56.201907 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.201825    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy-web\") pod \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") "
Oct 30 18:35:56.202390 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.201909    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f7ed38b7-0997-4cd8-9da0-41093596a6a0-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7ed38b7-0997-4cd8-9da0-41093596a6a0-alertmanager-trusted-ca-bundle\") pod \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\" (UID: \"f7ed38b7-0997-4cd8-9da0-41093596a6a0\") "
Oct 30 18:35:56.202390 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.202003    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7ed38b7-0997-4cd8-9da0-41093596a6a0-metrics-client-ca" pod "f7ed38b7-0997-4cd8-9da0-41093596a6a0" (UID: "f7ed38b7-0997-4cd8-9da0-41093596a6a0"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:35:56.202390 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.202372    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7ed38b7-0997-4cd8-9da0-41093596a6a0-metrics-client-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:56.204623 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.204593    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7ed38b7-0997-4cd8-9da0-41093596a6a0-alertmanager-main-db" pod "f7ed38b7-0997-4cd8-9da0-41093596a6a0" (UID: "f7ed38b7-0997-4cd8-9da0-41093596a6a0"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:35:56.204772 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.204617    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7ed38b7-0997-4cd8-9da0-41093596a6a0-alertmanager-trusted-ca-bundle" pod "f7ed38b7-0997-4cd8-9da0-41093596a6a0" (UID: "f7ed38b7-0997-4cd8-9da0-41093596a6a0"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:35:56.205265 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.205233    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy-metric" pod "f7ed38b7-0997-4cd8-9da0-41093596a6a0" (UID: "f7ed38b7-0997-4cd8-9da0-41093596a6a0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:35:56.205366 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.205275    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7ed38b7-0997-4cd8-9da0-41093596a6a0-config-out" pod "f7ed38b7-0997-4cd8-9da0-41093596a6a0" (UID: "f7ed38b7-0997-4cd8-9da0-41093596a6a0"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:35:56.205596 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.205572    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7ed38b7-0997-4cd8-9da0-41093596a6a0-kube-api-access-d4bcf" pod "f7ed38b7-0997-4cd8-9da0-41093596a6a0" (UID: "f7ed38b7-0997-4cd8-9da0-41093596a6a0"). InnerVolumeSpecName "kube-api-access-d4bcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:35:56.205663 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.205641    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-main-tls" pod "f7ed38b7-0997-4cd8-9da0-41093596a6a0" (UID: "f7ed38b7-0997-4cd8-9da0-41093596a6a0"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:35:56.205835 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.205806    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7ed38b7-0997-4cd8-9da0-41093596a6a0-tls-assets" pod "f7ed38b7-0997-4cd8-9da0-41093596a6a0" (UID: "f7ed38b7-0997-4cd8-9da0-41093596a6a0"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:35:56.205985 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.205901    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy-web" pod "f7ed38b7-0997-4cd8-9da0-41093596a6a0" (UID: "f7ed38b7-0997-4cd8-9da0-41093596a6a0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:35:56.206228 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.206203    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-config-volume" pod "f7ed38b7-0997-4cd8-9da0-41093596a6a0" (UID: "f7ed38b7-0997-4cd8-9da0-41093596a6a0"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:35:56.206780 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.206750    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy" pod "f7ed38b7-0997-4cd8-9da0-41093596a6a0" (UID: "f7ed38b7-0997-4cd8-9da0-41093596a6a0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:35:56.210452 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.210428    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-cluster-tls-config" pod "f7ed38b7-0997-4cd8-9da0-41093596a6a0" (UID: "f7ed38b7-0997-4cd8-9da0-41093596a6a0"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:35:56.216298 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.216279    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-web-config" pod "f7ed38b7-0997-4cd8-9da0-41093596a6a0" (UID: "f7ed38b7-0997-4cd8-9da0-41093596a6a0"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:35:56.303467 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.303434    5820 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7ed38b7-0997-4cd8-9da0-41093596a6a0-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:56.303467 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.303468    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4bcf\" (UniqueName: \"kubernetes.io/projected/f7ed38b7-0997-4cd8-9da0-41093596a6a0-kube-api-access-d4bcf\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:56.303673 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.303509    5820 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-web-config\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:56.303673 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.303528    5820 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-main-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:56.303673 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.303544    5820 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f7ed38b7-0997-4cd8-9da0-41093596a6a0-config-out\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:56.303673 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.303559    5820 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:56.303673 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.303573    5820 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f7ed38b7-0997-4cd8-9da0-41093596a6a0-alertmanager-main-db\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:56.303673 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.303586    5820 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:56.303673 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.303598    5820 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f7ed38b7-0997-4cd8-9da0-41093596a6a0-tls-assets\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:56.303673 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.303611    5820 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-cluster-tls-config\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:56.303673 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.303622    5820 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:56.303673 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.303632    5820 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7ed38b7-0997-4cd8-9da0-41093596a6a0-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:56.829532 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.829497    5820 generic.go:356] "Generic (PLEG): container finished" podID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerID="f7a1ef9cc808c0dc378f29e3ac1afd1b1833574b00dda97ae81661cc76c966c1" exitCode=0
Oct 30 18:35:56.829532 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.829522    5820 generic.go:356] "Generic (PLEG): container finished" podID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerID="43a340d025c45535e367a6aea3e41cdb52b3bde8dd27e6f372180e2e436f5680" exitCode=0
Oct 30 18:35:56.829730 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.829557    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7ed38b7-0997-4cd8-9da0-41093596a6a0","Type":"ContainerDied","Data":"f7a1ef9cc808c0dc378f29e3ac1afd1b1833574b00dda97ae81661cc76c966c1"}
Oct 30 18:35:56.829730 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.829578    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:56.829730 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.829611    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7ed38b7-0997-4cd8-9da0-41093596a6a0","Type":"ContainerDied","Data":"43a340d025c45535e367a6aea3e41cdb52b3bde8dd27e6f372180e2e436f5680"}
Oct 30 18:35:56.829730 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.829637    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7ed38b7-0997-4cd8-9da0-41093596a6a0","Type":"ContainerDied","Data":"b1e40d3592a73c7b18b1a77643b79ec547e87f7468511e27bab38a1ba707246d"}
Oct 30 18:35:56.829730 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.829667    5820 scope.go:122] "RemoveContainer" containerID="ecb8816873ee3bc29892301cdf67a02f36a68e05b30f5ce97a93854c9c8e4812"
Oct 30 18:35:56.840123 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.840096    5820 scope.go:122] "RemoveContainer" containerID="f7a1ef9cc808c0dc378f29e3ac1afd1b1833574b00dda97ae81661cc76c966c1"
Oct 30 18:35:56.847397 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.847383    5820 scope.go:122] "RemoveContainer" containerID="ad5dc45c6ed151741af41b6243ad6704239b5ba43a4a62af9cff03c6b57983bd"
Oct 30 18:35:56.853968 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.853929    5820 scope.go:122] "RemoveContainer" containerID="43a340d025c45535e367a6aea3e41cdb52b3bde8dd27e6f372180e2e436f5680"
Oct 30 18:35:56.860598 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.860583    5820 scope.go:122] "RemoveContainer" containerID="812c6cf749ef9fd286430ba652f60a717c618ee8d85654b467b446cd4fb6b8db"
Oct 30 18:35:56.867158 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.867139    5820 scope.go:122] "RemoveContainer" containerID="7fdca55ec4308700260efd57276b0518d1eb3d50e1eada4def6468a83c846095"
Oct 30 18:35:56.873868 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.873851    5820 scope.go:122] "RemoveContainer" containerID="62e3c9bf7e576b3b3d45f029b9b4d48e7d2bdf1e1c03a0103679146beaf27a5a"
Oct 30 18:35:56.880724 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.880709    5820 scope.go:122] "RemoveContainer" containerID="ecb8816873ee3bc29892301cdf67a02f36a68e05b30f5ce97a93854c9c8e4812"
Oct 30 18:35:56.880988 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:35:56.880943    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecb8816873ee3bc29892301cdf67a02f36a68e05b30f5ce97a93854c9c8e4812\": container with ID starting with ecb8816873ee3bc29892301cdf67a02f36a68e05b30f5ce97a93854c9c8e4812 not found: ID does not exist" containerID="ecb8816873ee3bc29892301cdf67a02f36a68e05b30f5ce97a93854c9c8e4812"
Oct 30 18:35:56.881034 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.881001    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecb8816873ee3bc29892301cdf67a02f36a68e05b30f5ce97a93854c9c8e4812"} err="failed to get container status \"ecb8816873ee3bc29892301cdf67a02f36a68e05b30f5ce97a93854c9c8e4812\": rpc error: code = NotFound desc = could not find container \"ecb8816873ee3bc29892301cdf67a02f36a68e05b30f5ce97a93854c9c8e4812\": container with ID starting with ecb8816873ee3bc29892301cdf67a02f36a68e05b30f5ce97a93854c9c8e4812 not found: ID does not exist"
Oct 30 18:35:56.881034 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.881028    5820 scope.go:122] "RemoveContainer" containerID="f7a1ef9cc808c0dc378f29e3ac1afd1b1833574b00dda97ae81661cc76c966c1"
Oct 30 18:35:56.881273 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:35:56.881257    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7a1ef9cc808c0dc378f29e3ac1afd1b1833574b00dda97ae81661cc76c966c1\": container with ID starting with f7a1ef9cc808c0dc378f29e3ac1afd1b1833574b00dda97ae81661cc76c966c1 not found: ID does not exist" containerID="f7a1ef9cc808c0dc378f29e3ac1afd1b1833574b00dda97ae81661cc76c966c1"
Oct 30 18:35:56.881317 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.881278    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7a1ef9cc808c0dc378f29e3ac1afd1b1833574b00dda97ae81661cc76c966c1"} err="failed to get container status \"f7a1ef9cc808c0dc378f29e3ac1afd1b1833574b00dda97ae81661cc76c966c1\": rpc error: code = NotFound desc = could not find container \"f7a1ef9cc808c0dc378f29e3ac1afd1b1833574b00dda97ae81661cc76c966c1\": container with ID starting with f7a1ef9cc808c0dc378f29e3ac1afd1b1833574b00dda97ae81661cc76c966c1 not found: ID does not exist"
Oct 30 18:35:56.881317 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.881295    5820 scope.go:122] "RemoveContainer" containerID="ad5dc45c6ed151741af41b6243ad6704239b5ba43a4a62af9cff03c6b57983bd"
Oct 30 18:35:56.881487 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:35:56.881472    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad5dc45c6ed151741af41b6243ad6704239b5ba43a4a62af9cff03c6b57983bd\": container with ID starting with ad5dc45c6ed151741af41b6243ad6704239b5ba43a4a62af9cff03c6b57983bd not found: ID does not exist" containerID="ad5dc45c6ed151741af41b6243ad6704239b5ba43a4a62af9cff03c6b57983bd"
Oct 30 18:35:56.881528 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.881491    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad5dc45c6ed151741af41b6243ad6704239b5ba43a4a62af9cff03c6b57983bd"} err="failed to get container status \"ad5dc45c6ed151741af41b6243ad6704239b5ba43a4a62af9cff03c6b57983bd\": rpc error: code = NotFound desc = could not find container \"ad5dc45c6ed151741af41b6243ad6704239b5ba43a4a62af9cff03c6b57983bd\": container with ID starting with ad5dc45c6ed151741af41b6243ad6704239b5ba43a4a62af9cff03c6b57983bd not found: ID does not exist"
Oct 30 18:35:56.881528 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.881503    5820 scope.go:122] "RemoveContainer" containerID="43a340d025c45535e367a6aea3e41cdb52b3bde8dd27e6f372180e2e436f5680"
Oct 30 18:35:56.881745 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:35:56.881731    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43a340d025c45535e367a6aea3e41cdb52b3bde8dd27e6f372180e2e436f5680\": container with ID starting with 43a340d025c45535e367a6aea3e41cdb52b3bde8dd27e6f372180e2e436f5680 not found: ID does not exist" containerID="43a340d025c45535e367a6aea3e41cdb52b3bde8dd27e6f372180e2e436f5680"
Oct 30 18:35:56.881786 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.881748    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43a340d025c45535e367a6aea3e41cdb52b3bde8dd27e6f372180e2e436f5680"} err="failed to get container status \"43a340d025c45535e367a6aea3e41cdb52b3bde8dd27e6f372180e2e436f5680\": rpc error: code = NotFound desc = could not find container \"43a340d025c45535e367a6aea3e41cdb52b3bde8dd27e6f372180e2e436f5680\": container with ID starting with 43a340d025c45535e367a6aea3e41cdb52b3bde8dd27e6f372180e2e436f5680 not found: ID does not exist"
Oct 30 18:35:56.881786 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.881759    5820 scope.go:122] "RemoveContainer" containerID="812c6cf749ef9fd286430ba652f60a717c618ee8d85654b467b446cd4fb6b8db"
Oct 30 18:35:56.881988 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:35:56.881968    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"812c6cf749ef9fd286430ba652f60a717c618ee8d85654b467b446cd4fb6b8db\": container with ID starting with 812c6cf749ef9fd286430ba652f60a717c618ee8d85654b467b446cd4fb6b8db not found: ID does not exist" containerID="812c6cf749ef9fd286430ba652f60a717c618ee8d85654b467b446cd4fb6b8db"
Oct 30 18:35:56.882030 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.881996    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"812c6cf749ef9fd286430ba652f60a717c618ee8d85654b467b446cd4fb6b8db"} err="failed to get container status \"812c6cf749ef9fd286430ba652f60a717c618ee8d85654b467b446cd4fb6b8db\": rpc error: code = NotFound desc = could not find container \"812c6cf749ef9fd286430ba652f60a717c618ee8d85654b467b446cd4fb6b8db\": container with ID starting with 812c6cf749ef9fd286430ba652f60a717c618ee8d85654b467b446cd4fb6b8db not found: ID does not exist"
Oct 30 18:35:56.882030 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.882012    5820 scope.go:122] "RemoveContainer" containerID="7fdca55ec4308700260efd57276b0518d1eb3d50e1eada4def6468a83c846095"
Oct 30 18:35:56.882287 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:35:56.882257    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fdca55ec4308700260efd57276b0518d1eb3d50e1eada4def6468a83c846095\": container with ID starting with 7fdca55ec4308700260efd57276b0518d1eb3d50e1eada4def6468a83c846095 not found: ID does not exist" containerID="7fdca55ec4308700260efd57276b0518d1eb3d50e1eada4def6468a83c846095"
Oct 30 18:35:56.882378 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.882290    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fdca55ec4308700260efd57276b0518d1eb3d50e1eada4def6468a83c846095"} err="failed to get container status \"7fdca55ec4308700260efd57276b0518d1eb3d50e1eada4def6468a83c846095\": rpc error: code = NotFound desc = could not find container \"7fdca55ec4308700260efd57276b0518d1eb3d50e1eada4def6468a83c846095\": container with ID starting with 7fdca55ec4308700260efd57276b0518d1eb3d50e1eada4def6468a83c846095 not found: ID does not exist"
Oct 30 18:35:56.882378 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.882310    5820 scope.go:122] "RemoveContainer" containerID="62e3c9bf7e576b3b3d45f029b9b4d48e7d2bdf1e1c03a0103679146beaf27a5a"
Oct 30 18:35:56.882587 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:35:56.882572    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62e3c9bf7e576b3b3d45f029b9b4d48e7d2bdf1e1c03a0103679146beaf27a5a\": container with ID starting with 62e3c9bf7e576b3b3d45f029b9b4d48e7d2bdf1e1c03a0103679146beaf27a5a not found: ID does not exist" containerID="62e3c9bf7e576b3b3d45f029b9b4d48e7d2bdf1e1c03a0103679146beaf27a5a"
Oct 30 18:35:56.882631 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.882590    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62e3c9bf7e576b3b3d45f029b9b4d48e7d2bdf1e1c03a0103679146beaf27a5a"} err="failed to get container status \"62e3c9bf7e576b3b3d45f029b9b4d48e7d2bdf1e1c03a0103679146beaf27a5a\": rpc error: code = NotFound desc = could not find container \"62e3c9bf7e576b3b3d45f029b9b4d48e7d2bdf1e1c03a0103679146beaf27a5a\": container with ID starting with 62e3c9bf7e576b3b3d45f029b9b4d48e7d2bdf1e1c03a0103679146beaf27a5a not found: ID does not exist"
Oct 30 18:35:56.882631 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.882605    5820 scope.go:122] "RemoveContainer" containerID="ecb8816873ee3bc29892301cdf67a02f36a68e05b30f5ce97a93854c9c8e4812"
Oct 30 18:35:56.882849 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.882822    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecb8816873ee3bc29892301cdf67a02f36a68e05b30f5ce97a93854c9c8e4812"} err="failed to get container status \"ecb8816873ee3bc29892301cdf67a02f36a68e05b30f5ce97a93854c9c8e4812\": rpc error: code = NotFound desc = could not find container \"ecb8816873ee3bc29892301cdf67a02f36a68e05b30f5ce97a93854c9c8e4812\": container with ID starting with ecb8816873ee3bc29892301cdf67a02f36a68e05b30f5ce97a93854c9c8e4812 not found: ID does not exist"
Oct 30 18:35:56.882887 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.882853    5820 scope.go:122] "RemoveContainer" containerID="f7a1ef9cc808c0dc378f29e3ac1afd1b1833574b00dda97ae81661cc76c966c1"
Oct 30 18:35:56.883111 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.883091    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7a1ef9cc808c0dc378f29e3ac1afd1b1833574b00dda97ae81661cc76c966c1"} err="failed to get container status \"f7a1ef9cc808c0dc378f29e3ac1afd1b1833574b00dda97ae81661cc76c966c1\": rpc error: code = NotFound desc = could not find container \"f7a1ef9cc808c0dc378f29e3ac1afd1b1833574b00dda97ae81661cc76c966c1\": container with ID starting with f7a1ef9cc808c0dc378f29e3ac1afd1b1833574b00dda97ae81661cc76c966c1 not found: ID does not exist"
Oct 30 18:35:56.883191 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.883112    5820 scope.go:122] "RemoveContainer" containerID="ad5dc45c6ed151741af41b6243ad6704239b5ba43a4a62af9cff03c6b57983bd"
Oct 30 18:35:56.883343 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.883327    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad5dc45c6ed151741af41b6243ad6704239b5ba43a4a62af9cff03c6b57983bd"} err="failed to get container status \"ad5dc45c6ed151741af41b6243ad6704239b5ba43a4a62af9cff03c6b57983bd\": rpc error: code = NotFound desc = could not find container \"ad5dc45c6ed151741af41b6243ad6704239b5ba43a4a62af9cff03c6b57983bd\": container with ID starting with ad5dc45c6ed151741af41b6243ad6704239b5ba43a4a62af9cff03c6b57983bd not found: ID does not exist"
Oct 30 18:35:56.883413 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.883344    5820 scope.go:122] "RemoveContainer" containerID="43a340d025c45535e367a6aea3e41cdb52b3bde8dd27e6f372180e2e436f5680"
Oct 30 18:35:56.883586 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.883568    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43a340d025c45535e367a6aea3e41cdb52b3bde8dd27e6f372180e2e436f5680"} err="failed to get container status \"43a340d025c45535e367a6aea3e41cdb52b3bde8dd27e6f372180e2e436f5680\": rpc error: code = NotFound desc = could not find container \"43a340d025c45535e367a6aea3e41cdb52b3bde8dd27e6f372180e2e436f5680\": container with ID starting with 43a340d025c45535e367a6aea3e41cdb52b3bde8dd27e6f372180e2e436f5680 not found: ID does not exist"
Oct 30 18:35:56.883640 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.883586    5820 scope.go:122] "RemoveContainer" containerID="812c6cf749ef9fd286430ba652f60a717c618ee8d85654b467b446cd4fb6b8db"
Oct 30 18:35:56.883803 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.883784    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"812c6cf749ef9fd286430ba652f60a717c618ee8d85654b467b446cd4fb6b8db"} err="failed to get container status \"812c6cf749ef9fd286430ba652f60a717c618ee8d85654b467b446cd4fb6b8db\": rpc error: code = NotFound desc = could not find container \"812c6cf749ef9fd286430ba652f60a717c618ee8d85654b467b446cd4fb6b8db\": container with ID starting with 812c6cf749ef9fd286430ba652f60a717c618ee8d85654b467b446cd4fb6b8db not found: ID does not exist"
Oct 30 18:35:56.883872 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.883804    5820 scope.go:122] "RemoveContainer" containerID="7fdca55ec4308700260efd57276b0518d1eb3d50e1eada4def6468a83c846095"
Oct 30 18:35:56.884085 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.884068    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fdca55ec4308700260efd57276b0518d1eb3d50e1eada4def6468a83c846095"} err="failed to get container status \"7fdca55ec4308700260efd57276b0518d1eb3d50e1eada4def6468a83c846095\": rpc error: code = NotFound desc = could not find container \"7fdca55ec4308700260efd57276b0518d1eb3d50e1eada4def6468a83c846095\": container with ID starting with 7fdca55ec4308700260efd57276b0518d1eb3d50e1eada4def6468a83c846095 not found: ID does not exist"
Oct 30 18:35:56.884135 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.884086    5820 scope.go:122] "RemoveContainer" containerID="62e3c9bf7e576b3b3d45f029b9b4d48e7d2bdf1e1c03a0103679146beaf27a5a"
Oct 30 18:35:56.884310 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.884291    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62e3c9bf7e576b3b3d45f029b9b4d48e7d2bdf1e1c03a0103679146beaf27a5a"} err="failed to get container status \"62e3c9bf7e576b3b3d45f029b9b4d48e7d2bdf1e1c03a0103679146beaf27a5a\": rpc error: code = NotFound desc = could not find container \"62e3c9bf7e576b3b3d45f029b9b4d48e7d2bdf1e1c03a0103679146beaf27a5a\": container with ID starting with 62e3c9bf7e576b3b3d45f029b9b4d48e7d2bdf1e1c03a0103679146beaf27a5a not found: ID does not exist"
Oct 30 18:35:56.995972 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:56.995922    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 18:35:57.045476 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.045443    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 18:35:57.046055 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.046027    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 18:35:57.046372 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.046356    5820 cpu_manager.go:414] "containerMap: removing container" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="kube-rbac-proxy"
Oct 30 18:35:57.046372 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.046374    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="kube-rbac-proxy"
Oct 30 18:35:57.046504 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.046390    5820 cpu_manager.go:414] "containerMap: removing container" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="kube-rbac-proxy-metric"
Oct 30 18:35:57.046504 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.046396    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="kube-rbac-proxy-metric"
Oct 30 18:35:57.046504 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.046407    5820 cpu_manager.go:414] "containerMap: removing container" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="init-config-reloader"
Oct 30 18:35:57.046504 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.046414    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="init-config-reloader"
Oct 30 18:35:57.046504 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.046433    5820 cpu_manager.go:414] "containerMap: removing container" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="prom-label-proxy"
Oct 30 18:35:57.046504 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.046440    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="prom-label-proxy"
Oct 30 18:35:57.046504 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.046450    5820 cpu_manager.go:414] "containerMap: removing container" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="kube-rbac-proxy-web"
Oct 30 18:35:57.046504 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.046455    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="kube-rbac-proxy-web"
Oct 30 18:35:57.046504 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.046466    5820 cpu_manager.go:414] "containerMap: removing container" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="config-reloader"
Oct 30 18:35:57.046504 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.046471    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="config-reloader"
Oct 30 18:35:57.046504 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.046477    5820 cpu_manager.go:414] "containerMap: removing container" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="alertmanager"
Oct 30 18:35:57.046504 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.046482    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="alertmanager"
Oct 30 18:35:57.046889 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.046525    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="prom-label-proxy"
Oct 30 18:35:57.046889 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.046533    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="kube-rbac-proxy-web"
Oct 30 18:35:57.046889 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.046540    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="config-reloader"
Oct 30 18:35:57.046889 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.046546    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="alertmanager"
Oct 30 18:35:57.046889 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.046551    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="kube-rbac-proxy"
Oct 30 18:35:57.046889 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.046558    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="kube-rbac-proxy-metric"
Oct 30 18:35:57.046889 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.046564    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" containerName="init-config-reloader"
Oct 30 18:35:57.049980 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.049944    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.107875 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.107793    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 18:35:57.107875 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.107801    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 18:35:57.108117 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.108010    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 18:35:57.108117 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.108046    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 18:35:57.108117 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.108084    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 18:35:57.108117 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.108087    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 18:35:57.108117 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.108088    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 18:35:57.108117 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.108099    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 18:35:57.108117 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.108091    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-gcq7x\""
Oct 30 18:35:57.110449 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.110430    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50ce9dff-2437-4686-89c5-f5354c0b547e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.110517 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.110457    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/50ce9dff-2437-4686-89c5-f5354c0b547e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.110517 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.110487    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/50ce9dff-2437-4686-89c5-f5354c0b547e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.110597 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.110548    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/50ce9dff-2437-4686-89c5-f5354c0b547e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.110597 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.110574    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/50ce9dff-2437-4686-89c5-f5354c0b547e-web-config\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.110692 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.110679    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/50ce9dff-2437-4686-89c5-f5354c0b547e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.110733 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.110702    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/50ce9dff-2437-4686-89c5-f5354c0b547e-config-volume\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.110733 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.110719    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/50ce9dff-2437-4686-89c5-f5354c0b547e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.110803 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.110736    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50ce9dff-2437-4686-89c5-f5354c0b547e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.110853 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.110837    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwntt\" (UniqueName: \"kubernetes.io/projected/50ce9dff-2437-4686-89c5-f5354c0b547e-kube-api-access-zwntt\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.110894 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.110867    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/50ce9dff-2437-4686-89c5-f5354c0b547e-config-out\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.110894 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.110887    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/50ce9dff-2437-4686-89c5-f5354c0b547e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.110979 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.110905    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/50ce9dff-2437-4686-89c5-f5354c0b547e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.157502 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.157482    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 18:35:57.211977 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.211935    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/50ce9dff-2437-4686-89c5-f5354c0b547e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.212088 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.211986    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/50ce9dff-2437-4686-89c5-f5354c0b547e-config-volume\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.212088 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.212013    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/50ce9dff-2437-4686-89c5-f5354c0b547e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.212088 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.212034    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50ce9dff-2437-4686-89c5-f5354c0b547e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.212088 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.212061    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwntt\" (UniqueName: \"kubernetes.io/projected/50ce9dff-2437-4686-89c5-f5354c0b547e-kube-api-access-zwntt\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.212276 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.212089    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/50ce9dff-2437-4686-89c5-f5354c0b547e-config-out\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.212276 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.212109    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/50ce9dff-2437-4686-89c5-f5354c0b547e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.212276 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.212196    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/50ce9dff-2437-4686-89c5-f5354c0b547e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.212276 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.212249    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50ce9dff-2437-4686-89c5-f5354c0b547e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.212510 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.212276    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/50ce9dff-2437-4686-89c5-f5354c0b547e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.212510 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.212299    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/50ce9dff-2437-4686-89c5-f5354c0b547e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.212510 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.212339    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/50ce9dff-2437-4686-89c5-f5354c0b547e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.213582 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.213017    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50ce9dff-2437-4686-89c5-f5354c0b547e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.213582 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.213074    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/50ce9dff-2437-4686-89c5-f5354c0b547e-web-config\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.213582 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.213479    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50ce9dff-2437-4686-89c5-f5354c0b547e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.213813 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.213588    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/50ce9dff-2437-4686-89c5-f5354c0b547e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.215337 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.215304    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/50ce9dff-2437-4686-89c5-f5354c0b547e-config-out\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.215445 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.215385    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/50ce9dff-2437-4686-89c5-f5354c0b547e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.215569 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.215547    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/50ce9dff-2437-4686-89c5-f5354c0b547e-config-volume\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.215686 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.215669    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/50ce9dff-2437-4686-89c5-f5354c0b547e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.216028 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.216008    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/50ce9dff-2437-4686-89c5-f5354c0b547e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.216207 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.216184    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/50ce9dff-2437-4686-89c5-f5354c0b547e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.216207 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.216198    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/50ce9dff-2437-4686-89c5-f5354c0b547e-web-config\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.216579 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.216563    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/50ce9dff-2437-4686-89c5-f5354c0b547e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.217287 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.217269    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/50ce9dff-2437-4686-89c5-f5354c0b547e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.275435 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.275411    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwntt\" (UniqueName: \"kubernetes.io/projected/50ce9dff-2437-4686-89c5-f5354c0b547e-kube-api-access-zwntt\") pod \"alertmanager-main-0\" (UID: \"50ce9dff-2437-4686-89c5-f5354c0b547e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.283974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.283925    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 18:35:57.366095 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.366030    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:35:57.434691 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.434662    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f7ed38b7-0997-4cd8-9da0-41093596a6a0" path="/var/lib/kubelet/pods/f7ed38b7-0997-4cd8-9da0-41093596a6a0/volumes"
Oct 30 18:35:57.793833 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.793758    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 18:35:57.797068 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:35:57.797048    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50ce9dff_2437_4686_89c5_f5354c0b547e.slice/crio-ee50af96920f1d8b5d9a512cd7a792f8039af49cd44e9867c4ab84dd3199b6c0 WatchSource:0}: Error finding container ee50af96920f1d8b5d9a512cd7a792f8039af49cd44e9867c4ab84dd3199b6c0: Status 404 returned error can't find the container with id ee50af96920f1d8b5d9a512cd7a792f8039af49cd44e9867c4ab84dd3199b6c0
Oct 30 18:35:57.834263 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:57.834231    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50ce9dff-2437-4686-89c5-f5354c0b547e","Type":"ContainerStarted","Data":"ee50af96920f1d8b5d9a512cd7a792f8039af49cd44e9867c4ab84dd3199b6c0"}
Oct 30 18:35:58.839924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:58.839889    5820 generic.go:356] "Generic (PLEG): container finished" podID="50ce9dff-2437-4686-89c5-f5354c0b547e" containerID="f287b16523279fe5231bbdd82f9ec6873a9f336ca98cca506bf0a6b6063c5155" exitCode=0
Oct 30 18:35:58.840439 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:58.839984    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50ce9dff-2437-4686-89c5-f5354c0b547e","Type":"ContainerDied","Data":"f287b16523279fe5231bbdd82f9ec6873a9f336ca98cca506bf0a6b6063c5155"}
Oct 30 18:35:59.509489 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:59.509409    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:59.509489 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:59.509471    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:59.513859 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:59.513841    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:59.847154 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:59.847119    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50ce9dff-2437-4686-89c5-f5354c0b547e","Type":"ContainerStarted","Data":"3611625ae9568177a372a13f9cfb0ac44ef879d58a2a12dff7f16ebbf73a5553"}
Oct 30 18:35:59.847642 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:59.847200    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50ce9dff-2437-4686-89c5-f5354c0b547e","Type":"ContainerStarted","Data":"f38958162a7ffa1e5ec363e99afbc51132a274cf7c59ab7f6de101c30e88f95e"}
Oct 30 18:35:59.847642 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:59.847243    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50ce9dff-2437-4686-89c5-f5354c0b547e","Type":"ContainerStarted","Data":"e972d87c04e02d35d3f4921e45d90514d16710b9fe276e9659a32b5b9becc026"}
Oct 30 18:35:59.847642 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:59.847284    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50ce9dff-2437-4686-89c5-f5354c0b547e","Type":"ContainerStarted","Data":"6b19c59a46a4c7f8ec0056094f064549b148c5da1b42f1e630cccac104d49277"}
Oct 30 18:35:59.847642 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:59.847325    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50ce9dff-2437-4686-89c5-f5354c0b547e","Type":"ContainerStarted","Data":"ef7988a4ee73d47369f67186044e07e2a34d78fa210f3e784c2fdbbca06e3c27"}
Oct 30 18:35:59.847642 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:59.847364    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50ce9dff-2437-4686-89c5-f5354c0b547e","Type":"ContainerStarted","Data":"c313a992ef1edbfde3288f523db09dd75c3757028bc11c00063f8311a9f681cc"}
Oct 30 18:35:59.851073 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:59.851049    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f68b76cbb-tr2kf"
Oct 30 18:35:59.884475 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:59.884449    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:35:59.984478 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:35:59.984426    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.976007196 podStartE2EDuration="3.984411303s" totalImagesPullingTime="0s" totalInitContainerRuntime="8.404107ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:35:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:35:59.983847458 +0000 UTC m=+80.695596155" watchObservedRunningTime="2026-10-30 18:35:59.984411303 +0000 UTC m=+80.696160001"
Oct 30 18:36:00.077580 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:00.077540    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:00.134686 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:00.134606    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76b7c98f9-cpqf9"]
Oct 30 18:36:03.154204 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:03.154167    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:03.333322 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:03.333271    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:06.351465 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:06.351429    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:06.577877 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:06.577843    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:08.202658 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:36:08.202619    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:36:25.154645 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.154548    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-76b7c98f9-cpqf9" podUID="37a53409-387e-4585-992f-ebbbc6c3b437" containerName="console" containerID="cri-o://07c1e2cb24e7ab8c9774cca17b1adb8e562d6e3d2886ddae7acec7dbe84fa36b" gracePeriod=15
Oct 30 18:36:25.400991 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.400968    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:36:25.453030 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.452944    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-trusted-ca-bundle\") pod \"37a53409-387e-4585-992f-ebbbc6c3b437\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") "
Oct 30 18:36:25.453030 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.453014    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-oauth-serving-cert\") pod \"37a53409-387e-4585-992f-ebbbc6c3b437\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") "
Oct 30 18:36:25.453223 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.453052    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/37a53409-387e-4585-992f-ebbbc6c3b437-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37a53409-387e-4585-992f-ebbbc6c3b437-console-oauth-config\") pod \"37a53409-387e-4585-992f-ebbbc6c3b437\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") "
Oct 30 18:36:25.453223 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.453085    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/37a53409-387e-4585-992f-ebbbc6c3b437-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37a53409-387e-4585-992f-ebbbc6c3b437-console-serving-cert\") pod \"37a53409-387e-4585-992f-ebbbc6c3b437\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") "
Oct 30 18:36:25.453223 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.453109    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-service-ca\" (UniqueName: \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-service-ca\") pod \"37a53409-387e-4585-992f-ebbbc6c3b437\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") "
Oct 30 18:36:25.453223 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.453146    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37a53409-387e-4585-992f-ebbbc6c3b437-tmp\" (UniqueName: \"kubernetes.io/empty-dir/37a53409-387e-4585-992f-ebbbc6c3b437-tmp\") pod \"37a53409-387e-4585-992f-ebbbc6c3b437\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") "
Oct 30 18:36:25.453223 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.453205    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37a53409-387e-4585-992f-ebbbc6c3b437-kube-api-access-8thzl\" (UniqueName: \"kubernetes.io/projected/37a53409-387e-4585-992f-ebbbc6c3b437-kube-api-access-8thzl\") pod \"37a53409-387e-4585-992f-ebbbc6c3b437\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") "
Oct 30 18:36:25.453466 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.453262    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-console-config\" (UniqueName: \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-console-config\") pod \"37a53409-387e-4585-992f-ebbbc6c3b437\" (UID: \"37a53409-387e-4585-992f-ebbbc6c3b437\") "
Oct 30 18:36:25.453466 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.453429    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-trusted-ca-bundle" pod "37a53409-387e-4585-992f-ebbbc6c3b437" (UID: "37a53409-387e-4585-992f-ebbbc6c3b437"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:36:25.453466 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.453435    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-oauth-serving-cert" pod "37a53409-387e-4585-992f-ebbbc6c3b437" (UID: "37a53409-387e-4585-992f-ebbbc6c3b437"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:36:25.453621 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.453576    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37a53409-387e-4585-992f-ebbbc6c3b437-tmp" pod "37a53409-387e-4585-992f-ebbbc6c3b437" (UID: "37a53409-387e-4585-992f-ebbbc6c3b437"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:36:25.453783 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.453760    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-service-ca" pod "37a53409-387e-4585-992f-ebbbc6c3b437" (UID: "37a53409-387e-4585-992f-ebbbc6c3b437"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:36:25.453980 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.453799    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-console-config" pod "37a53409-387e-4585-992f-ebbbc6c3b437" (UID: "37a53409-387e-4585-992f-ebbbc6c3b437"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:36:25.455463 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.455436    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37a53409-387e-4585-992f-ebbbc6c3b437-console-oauth-config" pod "37a53409-387e-4585-992f-ebbbc6c3b437" (UID: "37a53409-387e-4585-992f-ebbbc6c3b437"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:36:25.455568 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.455494    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37a53409-387e-4585-992f-ebbbc6c3b437-kube-api-access-8thzl" pod "37a53409-387e-4585-992f-ebbbc6c3b437" (UID: "37a53409-387e-4585-992f-ebbbc6c3b437"). InnerVolumeSpecName "kube-api-access-8thzl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:36:25.455568 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.455509    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37a53409-387e-4585-992f-ebbbc6c3b437-console-serving-cert" pod "37a53409-387e-4585-992f-ebbbc6c3b437" (UID: "37a53409-387e-4585-992f-ebbbc6c3b437"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:36:25.554236 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.554198    5820 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-console-config\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:36:25.554236 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.554229    5820 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-trusted-ca-bundle\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:36:25.554236 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.554242    5820 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-oauth-serving-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:36:25.554465 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.554256    5820 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37a53409-387e-4585-992f-ebbbc6c3b437-console-oauth-config\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:36:25.554465 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.554268    5820 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37a53409-387e-4585-992f-ebbbc6c3b437-console-serving-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:36:25.554465 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.554278    5820 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37a53409-387e-4585-992f-ebbbc6c3b437-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:36:25.554465 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.554289    5820 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37a53409-387e-4585-992f-ebbbc6c3b437-tmp\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:36:25.554465 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.554301    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8thzl\" (UniqueName: \"kubernetes.io/projected/37a53409-387e-4585-992f-ebbbc6c3b437-kube-api-access-8thzl\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:36:25.938827 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.938794    5820 generic.go:356] "Generic (PLEG): container finished" podID="37a53409-387e-4585-992f-ebbbc6c3b437" containerID="07c1e2cb24e7ab8c9774cca17b1adb8e562d6e3d2886ddae7acec7dbe84fa36b" exitCode=0
Oct 30 18:36:25.939033 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.938844    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76b7c98f9-cpqf9"
Oct 30 18:36:25.939033 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.938861    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76b7c98f9-cpqf9" event={"ID":"37a53409-387e-4585-992f-ebbbc6c3b437","Type":"ContainerDied","Data":"07c1e2cb24e7ab8c9774cca17b1adb8e562d6e3d2886ddae7acec7dbe84fa36b"}
Oct 30 18:36:25.939033 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.938900    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76b7c98f9-cpqf9" event={"ID":"37a53409-387e-4585-992f-ebbbc6c3b437","Type":"ContainerDied","Data":"c1bbf97788089409bbf24b2d62c1f1139cd5cac3bedbf9440ccb221dc34d8434"}
Oct 30 18:36:25.939033 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.938919    5820 scope.go:122] "RemoveContainer" containerID="07c1e2cb24e7ab8c9774cca17b1adb8e562d6e3d2886ddae7acec7dbe84fa36b"
Oct 30 18:36:25.947486 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.947465    5820 scope.go:122] "RemoveContainer" containerID="07c1e2cb24e7ab8c9774cca17b1adb8e562d6e3d2886ddae7acec7dbe84fa36b"
Oct 30 18:36:25.947750 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:36:25.947733    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07c1e2cb24e7ab8c9774cca17b1adb8e562d6e3d2886ddae7acec7dbe84fa36b\": container with ID starting with 07c1e2cb24e7ab8c9774cca17b1adb8e562d6e3d2886ddae7acec7dbe84fa36b not found: ID does not exist" containerID="07c1e2cb24e7ab8c9774cca17b1adb8e562d6e3d2886ddae7acec7dbe84fa36b"
Oct 30 18:36:25.947801 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:25.947759    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07c1e2cb24e7ab8c9774cca17b1adb8e562d6e3d2886ddae7acec7dbe84fa36b"} err="failed to get container status \"07c1e2cb24e7ab8c9774cca17b1adb8e562d6e3d2886ddae7acec7dbe84fa36b\": rpc error: code = NotFound desc = could not find container \"07c1e2cb24e7ab8c9774cca17b1adb8e562d6e3d2886ddae7acec7dbe84fa36b\": container with ID starting with 07c1e2cb24e7ab8c9774cca17b1adb8e562d6e3d2886ddae7acec7dbe84fa36b not found: ID does not exist"
Oct 30 18:36:26.102178 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:26.102146    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76b7c98f9-cpqf9"]
Oct 30 18:36:26.152190 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:26.152143    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76b7c98f9-cpqf9"]
Oct 30 18:36:27.433707 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:27.433675    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="37a53409-387e-4585-992f-ebbbc6c3b437" path="/var/lib/kubelet/pods/37a53409-387e-4585-992f-ebbbc6c3b437/volumes"
Oct 30 18:36:38.202103 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:36:38.202066    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:36:46.801801 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:36:46.801761    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:08.202109 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:37:08.202073    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:37:18.434326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:18.434295    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:21.448667 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:21.448622    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:38.201637 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:37:38.201596    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:37:38.863466 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:38.863433    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:39.128452 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:39.128360    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:40.292355 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.292323    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb"]
Oct 30 18:37:40.292727 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.292648    5820 cpu_manager.go:414] "containerMap: removing container" podUID="37a53409-387e-4585-992f-ebbbc6c3b437" containerName="console"
Oct 30 18:37:40.292727 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.292659    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="37a53409-387e-4585-992f-ebbbc6c3b437" containerName="console"
Oct 30 18:37:40.292727 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.292727    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="37a53409-387e-4585-992f-ebbbc6c3b437" containerName="console"
Oct 30 18:37:40.295737 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.295722    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb"
Oct 30 18:37:40.329677 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.329638    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c5b13d2-ee83-43ef-95a9-880efafa8eec-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb\" (UID: \"6c5b13d2-ee83-43ef-95a9-880efafa8eec\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb"
Oct 30 18:37:40.329930 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.329911    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcljd\" (UniqueName: \"kubernetes.io/projected/6c5b13d2-ee83-43ef-95a9-880efafa8eec-kube-api-access-lcljd\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb\" (UID: \"6c5b13d2-ee83-43ef-95a9-880efafa8eec\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb"
Oct 30 18:37:40.330183 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.330151    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c5b13d2-ee83-43ef-95a9-880efafa8eec-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb\" (UID: \"6c5b13d2-ee83-43ef-95a9-880efafa8eec\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb"
Oct 30 18:37:40.349108 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.349079    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 18:37:40.398858 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.398824    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 18:37:40.398858 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.398851    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-zk7vp\""
Oct 30 18:37:40.399844 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.399827    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb"]
Oct 30 18:37:40.431454 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.431418    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c5b13d2-ee83-43ef-95a9-880efafa8eec-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb\" (UID: \"6c5b13d2-ee83-43ef-95a9-880efafa8eec\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb"
Oct 30 18:37:40.431613 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.431461    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcljd\" (UniqueName: \"kubernetes.io/projected/6c5b13d2-ee83-43ef-95a9-880efafa8eec-kube-api-access-lcljd\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb\" (UID: \"6c5b13d2-ee83-43ef-95a9-880efafa8eec\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb"
Oct 30 18:37:40.431613 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.431506    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c5b13d2-ee83-43ef-95a9-880efafa8eec-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb\" (UID: \"6c5b13d2-ee83-43ef-95a9-880efafa8eec\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb"
Oct 30 18:37:40.431879 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.431858    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c5b13d2-ee83-43ef-95a9-880efafa8eec-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb\" (UID: \"6c5b13d2-ee83-43ef-95a9-880efafa8eec\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb"
Oct 30 18:37:40.431915 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.431866    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c5b13d2-ee83-43ef-95a9-880efafa8eec-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb\" (UID: \"6c5b13d2-ee83-43ef-95a9-880efafa8eec\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb"
Oct 30 18:37:40.452769 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.452737    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh"]
Oct 30 18:37:40.455987 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.455972    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh"
Oct 30 18:37:40.503761 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.503733    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcljd\" (UniqueName: \"kubernetes.io/projected/6c5b13d2-ee83-43ef-95a9-880efafa8eec-kube-api-access-lcljd\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb\" (UID: \"6c5b13d2-ee83-43ef-95a9-880efafa8eec\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb"
Oct 30 18:37:40.509210 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.509188    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-hkslx\""
Oct 30 18:37:40.517202 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.517178    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh"]
Oct 30 18:37:40.532660 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.532637    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49bd3da2-e0b8-4dd9-a904-626b88b3afdf-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh\" (UID: \"49bd3da2-e0b8-4dd9-a904-626b88b3afdf\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh"
Oct 30 18:37:40.532784 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.532724    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf8xq\" (UniqueName: \"kubernetes.io/projected/49bd3da2-e0b8-4dd9-a904-626b88b3afdf-kube-api-access-tf8xq\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh\" (UID: \"49bd3da2-e0b8-4dd9-a904-626b88b3afdf\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh"
Oct 30 18:37:40.532993 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.532961    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49bd3da2-e0b8-4dd9-a904-626b88b3afdf-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh\" (UID: \"49bd3da2-e0b8-4dd9-a904-626b88b3afdf\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh"
Oct 30 18:37:40.577924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.577900    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"]
Oct 30 18:37:40.581110 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.581095    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.610940 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.610916    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb"
Oct 30 18:37:40.633547 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.633524    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf8xq\" (UniqueName: \"kubernetes.io/projected/49bd3da2-e0b8-4dd9-a904-626b88b3afdf-kube-api-access-tf8xq\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh\" (UID: \"49bd3da2-e0b8-4dd9-a904-626b88b3afdf\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh"
Oct 30 18:37:40.633649 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.633554    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f8ec0830-869f-43d4-a461-e56b64b7a4ea-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-lz85p\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.633649 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.633580    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-lz85p\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.633649 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.633598    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8ec0830-869f-43d4-a461-e56b64b7a4ea-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-lz85p\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.633649 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.633625    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l64xr\" (UniqueName: \"kubernetes.io/projected/f8ec0830-869f-43d4-a461-e56b64b7a4ea-kube-api-access-l64xr\") pod \"istiod-openshift-gateway-7876bfcb74-lz85p\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.633649 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.633645    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49bd3da2-e0b8-4dd9-a904-626b88b3afdf-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh\" (UID: \"49bd3da2-e0b8-4dd9-a904-626b88b3afdf\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh"
Oct 30 18:37:40.633842 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.633814    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-lz85p\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.633888 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.633857    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-lz85p\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.633940 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.633894    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49bd3da2-e0b8-4dd9-a904-626b88b3afdf-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh\" (UID: \"49bd3da2-e0b8-4dd9-a904-626b88b3afdf\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh"
Oct 30 18:37:40.634014 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.633992    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-lz85p\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.634130 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.634113    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49bd3da2-e0b8-4dd9-a904-626b88b3afdf-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh\" (UID: \"49bd3da2-e0b8-4dd9-a904-626b88b3afdf\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh"
Oct 30 18:37:40.634201 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.634183    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49bd3da2-e0b8-4dd9-a904-626b88b3afdf-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh\" (UID: \"49bd3da2-e0b8-4dd9-a904-626b88b3afdf\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh"
Oct 30 18:37:40.635589 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.635571    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 18:37:40.635589 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.635584    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jhrl9\""
Oct 30 18:37:40.640191 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.640168    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 18:37:40.644270 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.644248    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"]
Oct 30 18:37:40.653975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.653914    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:40.697623 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.697595    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf8xq\" (UniqueName: \"kubernetes.io/projected/49bd3da2-e0b8-4dd9-a904-626b88b3afdf-kube-api-access-tf8xq\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh\" (UID: \"49bd3da2-e0b8-4dd9-a904-626b88b3afdf\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh"
Oct 30 18:37:40.734545 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.734516    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-lz85p\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.734675 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.734556    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f8ec0830-869f-43d4-a461-e56b64b7a4ea-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-lz85p\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.734675 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.734583    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-lz85p\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.734675 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.734610    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8ec0830-869f-43d4-a461-e56b64b7a4ea-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-lz85p\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.734675 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.734656    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l64xr\" (UniqueName: \"kubernetes.io/projected/f8ec0830-869f-43d4-a461-e56b64b7a4ea-kube-api-access-l64xr\") pod \"istiod-openshift-gateway-7876bfcb74-lz85p\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.734866 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.734710    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-lz85p\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.734866 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.734732    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-lz85p\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.734992 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.734901    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 18:37:40.735057 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.735028    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 18:37:40.735243 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.735220    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-lz85p\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.736911 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.736894    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8ec0830-869f-43d4-a461-e56b64b7a4ea-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-lz85p\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.736993 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.736937    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f8ec0830-869f-43d4-a461-e56b64b7a4ea-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-lz85p\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.740620 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.740603    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 18:37:40.740732 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.740719    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 18:37:40.747104 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.746937    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-lz85p\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.747507 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.747474    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-lz85p\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.771885 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.771864    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh"
Oct 30 18:37:40.796075 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.796053    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-lz85p\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.796368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.796349    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l64xr\" (UniqueName: \"kubernetes.io/projected/f8ec0830-869f-43d4-a461-e56b64b7a4ea-kube-api-access-l64xr\") pod \"istiod-openshift-gateway-7876bfcb74-lz85p\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.867191 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.867109    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:40.893032 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.893010    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:40.988353 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.988328    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb"]
Oct 30 18:37:40.991515 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:37:40.991485    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c5b13d2_ee83_43ef_95a9_880efafa8eec.slice/crio-72972055f619c5a986c4db5531bbdd0f04097c7eb356b5c3770d1922d7b27aef WatchSource:0}: Error finding container 72972055f619c5a986c4db5531bbdd0f04097c7eb356b5c3770d1922d7b27aef: Status 404 returned error can't find the container with id 72972055f619c5a986c4db5531bbdd0f04097c7eb356b5c3770d1922d7b27aef
Oct 30 18:37:40.993642 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:40.993623    5820 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:37:41.142792 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:41.142707    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh"]
Oct 30 18:37:41.145148 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:37:41.145124    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49bd3da2_e0b8_4dd9_a904_626b88b3afdf.slice/crio-7d5014abf9e7f9c2eebfbf88febfb2d9a2032b62cc85e21d12af40aefd8cfa7a WatchSource:0}: Error finding container 7d5014abf9e7f9c2eebfbf88febfb2d9a2032b62cc85e21d12af40aefd8cfa7a: Status 404 returned error can't find the container with id 7d5014abf9e7f9c2eebfbf88febfb2d9a2032b62cc85e21d12af40aefd8cfa7a
Oct 30 18:37:41.215279 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:41.215245    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh" event={"ID":"49bd3da2-e0b8-4dd9-a904-626b88b3afdf","Type":"ContainerStarted","Data":"7d5014abf9e7f9c2eebfbf88febfb2d9a2032b62cc85e21d12af40aefd8cfa7a"}
Oct 30 18:37:41.216342 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:41.216322    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb" event={"ID":"6c5b13d2-ee83-43ef-95a9-880efafa8eec","Type":"ContainerStarted","Data":"72972055f619c5a986c4db5531bbdd0f04097c7eb356b5c3770d1922d7b27aef"}
Oct 30 18:37:41.272602 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:41.272582    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"]
Oct 30 18:37:41.277470 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:37:41.277447    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8ec0830_869f_43d4_a461_e56b64b7a4ea.slice/crio-f8d9d8dcefa33b16114a2e6ec880c169c86957a816c443996736d153a9a76ce7 WatchSource:0}: Error finding container f8d9d8dcefa33b16114a2e6ec880c169c86957a816c443996736d153a9a76ce7: Status 404 returned error can't find the container with id f8d9d8dcefa33b16114a2e6ec880c169c86957a816c443996736d153a9a76ce7
Oct 30 18:37:42.048522 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:42.048485    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:42.222823 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:42.222779    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p" event={"ID":"f8ec0830-869f-43d4-a461-e56b64b7a4ea","Type":"ContainerStarted","Data":"f8d9d8dcefa33b16114a2e6ec880c169c86957a816c443996736d153a9a76ce7"}
Oct 30 18:37:45.943062 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:45.942598    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:37:45.943062 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:45.942671    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:37:46.245409 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:46.244461    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p" event={"ID":"f8ec0830-869f-43d4-a461-e56b64b7a4ea","Type":"ContainerStarted","Data":"f3ca58e89e93667fee1b54ddb1e8fa770637c1bcdcf80391d35a72add205eda4"}
Oct 30 18:37:46.245409 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:46.245316    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:46.465514 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:46.465458    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p" podStartSLOduration=1.8031106110000001 podStartE2EDuration="6.465437453s" totalImagesPullingTime="4.662326842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:46.415477412 +0000 UTC m=+187.127226109" watchObservedRunningTime="2026-10-30 18:37:46.465437453 +0000 UTC m=+187.177186153"
Oct 30 18:37:47.164893 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:47.164856    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:47.251999 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:47.251799    5820 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-lz85p container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 18:37:47.251999 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:47.251851    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p" podUID="f8ec0830-869f-43d4-a461-e56b64b7a4ea" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:37:47.636014 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:47.634824    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:47.824193 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:47.824136    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:48.254862 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:48.254832    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:37:48.270199 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:48.270042    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:48.543074 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:48.543043    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:48.721852 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:48.721819    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:49.259036 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:49.259007    5820 generic.go:356] "Generic (PLEG): container finished" podID="49bd3da2-e0b8-4dd9-a904-626b88b3afdf" containerID="c80dd79d1ed5ccd61d763556e72521c97186e2fc90051968d1fbcb1542b6e13d" exitCode=0
Oct 30 18:37:49.259492 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:49.259081    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh" event={"ID":"49bd3da2-e0b8-4dd9-a904-626b88b3afdf","Type":"ContainerDied","Data":"c80dd79d1ed5ccd61d763556e72521c97186e2fc90051968d1fbcb1542b6e13d"}
Oct 30 18:37:49.260705 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:49.260686    5820 generic.go:356] "Generic (PLEG): container finished" podID="6c5b13d2-ee83-43ef-95a9-880efafa8eec" containerID="e2014a71dcc22c787f8210c33a2b86d37df7fe0303f8bfa927cb241682f9de19" exitCode=0
Oct 30 18:37:49.260815 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:49.260791    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb" event={"ID":"6c5b13d2-ee83-43ef-95a9-880efafa8eec","Type":"ContainerDied","Data":"e2014a71dcc22c787f8210c33a2b86d37df7fe0303f8bfa927cb241682f9de19"}
Oct 30 18:37:50.266601 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:50.266571    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh" event={"ID":"49bd3da2-e0b8-4dd9-a904-626b88b3afdf","Type":"ContainerStarted","Data":"ec6d0952676f1610ad17b0d2db6d4eb7082e3b39fa6886adedc3d0ee8921d0e7"}
Oct 30 18:37:51.273889 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:51.273850    5820 generic.go:356] "Generic (PLEG): container finished" podID="49bd3da2-e0b8-4dd9-a904-626b88b3afdf" containerID="ec6d0952676f1610ad17b0d2db6d4eb7082e3b39fa6886adedc3d0ee8921d0e7" exitCode=0
Oct 30 18:37:51.274389 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:51.273943    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh" event={"ID":"49bd3da2-e0b8-4dd9-a904-626b88b3afdf","Type":"ContainerDied","Data":"ec6d0952676f1610ad17b0d2db6d4eb7082e3b39fa6886adedc3d0ee8921d0e7"}
Oct 30 18:37:52.128674 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:52.128628    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:53.285877 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:53.285839    5820 generic.go:356] "Generic (PLEG): container finished" podID="6c5b13d2-ee83-43ef-95a9-880efafa8eec" containerID="2110fda969b03a29a440a79e500277313f44cb5d3f0b908c9a9af0448c3813f2" exitCode=0
Oct 30 18:37:53.286299 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:53.285934    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb" event={"ID":"6c5b13d2-ee83-43ef-95a9-880efafa8eec","Type":"ContainerDied","Data":"2110fda969b03a29a440a79e500277313f44cb5d3f0b908c9a9af0448c3813f2"}
Oct 30 18:37:53.356774 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:53.356746    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:53.581203 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:53.581154    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:59.314118 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:59.314081    5820 generic.go:356] "Generic (PLEG): container finished" podID="49bd3da2-e0b8-4dd9-a904-626b88b3afdf" containerID="3b809f556c81823bf5f4566883e5ad25dab80903d6d0bd6e146c6c9645edcd41" exitCode=0
Oct 30 18:37:59.314560 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:37:59.314164    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh" event={"ID":"49bd3da2-e0b8-4dd9-a904-626b88b3afdf","Type":"ContainerDied","Data":"3b809f556c81823bf5f4566883e5ad25dab80903d6d0bd6e146c6c9645edcd41"}
Oct 30 18:38:00.320186 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:00.320153    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb" event={"ID":"6c5b13d2-ee83-43ef-95a9-880efafa8eec","Type":"ContainerStarted","Data":"75aac5e1b4dc4c16e27b3b93b7c13e3f2059092f1e25126440cc985ee1cfe3b8"}
Oct 30 18:38:00.438276 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:00.438211    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb" podStartSLOduration=2.460432333 podStartE2EDuration="20.438171196s" totalImagesPullingTime="17.977738863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:00.437193428 +0000 UTC m=+201.148942127" watchObservedRunningTime="2026-10-30 18:38:00.438171196 +0000 UTC m=+201.149919893"
Oct 30 18:38:00.440494 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:00.440471    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh"
Oct 30 18:38:00.511359 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:00.511331    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49bd3da2-e0b8-4dd9-a904-626b88b3afdf-util\" (UniqueName: \"kubernetes.io/empty-dir/49bd3da2-e0b8-4dd9-a904-626b88b3afdf-util\") pod \"49bd3da2-e0b8-4dd9-a904-626b88b3afdf\" (UID: \"49bd3da2-e0b8-4dd9-a904-626b88b3afdf\") "
Oct 30 18:38:00.511492 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:00.511404    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49bd3da2-e0b8-4dd9-a904-626b88b3afdf-kube-api-access-tf8xq\" (UniqueName: \"kubernetes.io/projected/49bd3da2-e0b8-4dd9-a904-626b88b3afdf-kube-api-access-tf8xq\") pod \"49bd3da2-e0b8-4dd9-a904-626b88b3afdf\" (UID: \"49bd3da2-e0b8-4dd9-a904-626b88b3afdf\") "
Oct 30 18:38:00.511492 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:00.511444    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49bd3da2-e0b8-4dd9-a904-626b88b3afdf-bundle\" (UniqueName: \"kubernetes.io/empty-dir/49bd3da2-e0b8-4dd9-a904-626b88b3afdf-bundle\") pod \"49bd3da2-e0b8-4dd9-a904-626b88b3afdf\" (UID: \"49bd3da2-e0b8-4dd9-a904-626b88b3afdf\") "
Oct 30 18:38:00.512117 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:00.512081    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49bd3da2-e0b8-4dd9-a904-626b88b3afdf-bundle" pod "49bd3da2-e0b8-4dd9-a904-626b88b3afdf" (UID: "49bd3da2-e0b8-4dd9-a904-626b88b3afdf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:00.513659 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:00.513640    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49bd3da2-e0b8-4dd9-a904-626b88b3afdf-kube-api-access-tf8xq" pod "49bd3da2-e0b8-4dd9-a904-626b88b3afdf" (UID: "49bd3da2-e0b8-4dd9-a904-626b88b3afdf"). InnerVolumeSpecName "kube-api-access-tf8xq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:38:00.516775 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:00.516751    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49bd3da2-e0b8-4dd9-a904-626b88b3afdf-util" pod "49bd3da2-e0b8-4dd9-a904-626b88b3afdf" (UID: "49bd3da2-e0b8-4dd9-a904-626b88b3afdf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:00.613061 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:00.613035    5820 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49bd3da2-e0b8-4dd9-a904-626b88b3afdf-util\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:00.613061 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:00.613060    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tf8xq\" (UniqueName: \"kubernetes.io/projected/49bd3da2-e0b8-4dd9-a904-626b88b3afdf-kube-api-access-tf8xq\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:00.613223 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:00.613069    5820 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49bd3da2-e0b8-4dd9-a904-626b88b3afdf-bundle\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:01.326668 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:01.326630    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh" event={"ID":"49bd3da2-e0b8-4dd9-a904-626b88b3afdf","Type":"ContainerDied","Data":"7d5014abf9e7f9c2eebfbf88febfb2d9a2032b62cc85e21d12af40aefd8cfa7a"}
Oct 30 18:38:01.327119 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:01.326693    5820 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d5014abf9e7f9c2eebfbf88febfb2d9a2032b62cc85e21d12af40aefd8cfa7a"
Oct 30 18:38:01.327119 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:01.326648    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be714hhbh"
Oct 30 18:38:01.328770 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:01.328738    5820 generic.go:356] "Generic (PLEG): container finished" podID="6c5b13d2-ee83-43ef-95a9-880efafa8eec" containerID="75aac5e1b4dc4c16e27b3b93b7c13e3f2059092f1e25126440cc985ee1cfe3b8" exitCode=0
Oct 30 18:38:01.328865 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:01.328793    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb" event={"ID":"6c5b13d2-ee83-43ef-95a9-880efafa8eec","Type":"ContainerDied","Data":"75aac5e1b4dc4c16e27b3b93b7c13e3f2059092f1e25126440cc985ee1cfe3b8"}
Oct 30 18:38:02.503044 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:02.503012    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb"
Oct 30 18:38:02.631884 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:02.631815    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c5b13d2-ee83-43ef-95a9-880efafa8eec-bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c5b13d2-ee83-43ef-95a9-880efafa8eec-bundle\") pod \"6c5b13d2-ee83-43ef-95a9-880efafa8eec\" (UID: \"6c5b13d2-ee83-43ef-95a9-880efafa8eec\") "
Oct 30 18:38:02.632036 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:02.631895    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c5b13d2-ee83-43ef-95a9-880efafa8eec-kube-api-access-lcljd\" (UniqueName: \"kubernetes.io/projected/6c5b13d2-ee83-43ef-95a9-880efafa8eec-kube-api-access-lcljd\") pod \"6c5b13d2-ee83-43ef-95a9-880efafa8eec\" (UID: \"6c5b13d2-ee83-43ef-95a9-880efafa8eec\") "
Oct 30 18:38:02.632036 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:02.631941    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c5b13d2-ee83-43ef-95a9-880efafa8eec-util\" (UniqueName: \"kubernetes.io/empty-dir/6c5b13d2-ee83-43ef-95a9-880efafa8eec-util\") pod \"6c5b13d2-ee83-43ef-95a9-880efafa8eec\" (UID: \"6c5b13d2-ee83-43ef-95a9-880efafa8eec\") "
Oct 30 18:38:02.632326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:02.632297    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c5b13d2-ee83-43ef-95a9-880efafa8eec-bundle" pod "6c5b13d2-ee83-43ef-95a9-880efafa8eec" (UID: "6c5b13d2-ee83-43ef-95a9-880efafa8eec"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:02.634178 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:02.634153    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c5b13d2-ee83-43ef-95a9-880efafa8eec-kube-api-access-lcljd" pod "6c5b13d2-ee83-43ef-95a9-880efafa8eec" (UID: "6c5b13d2-ee83-43ef-95a9-880efafa8eec"). InnerVolumeSpecName "kube-api-access-lcljd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:38:02.638395 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:02.638372    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c5b13d2-ee83-43ef-95a9-880efafa8eec-util" pod "6c5b13d2-ee83-43ef-95a9-880efafa8eec" (UID: "6c5b13d2-ee83-43ef-95a9-880efafa8eec"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:02.733167 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:02.733141    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcljd\" (UniqueName: \"kubernetes.io/projected/6c5b13d2-ee83-43ef-95a9-880efafa8eec-kube-api-access-lcljd\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:02.733167 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:02.733164    5820 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c5b13d2-ee83-43ef-95a9-880efafa8eec-util\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:02.733167 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:02.733174    5820 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c5b13d2-ee83-43ef-95a9-880efafa8eec-bundle\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:03.337504 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:03.337474    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb" event={"ID":"6c5b13d2-ee83-43ef-95a9-880efafa8eec","Type":"ContainerDied","Data":"72972055f619c5a986c4db5531bbdd0f04097c7eb356b5c3770d1922d7b27aef"}
Oct 30 18:38:03.337504 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:03.337507    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644dlmfb"
Oct 30 18:38:03.337682 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:03.337547    5820 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72972055f619c5a986c4db5531bbdd0f04097c7eb356b5c3770d1922d7b27aef"
Oct 30 18:38:08.201594 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:38:08.201554    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:38:21.896740 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:21.896705    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:22.075085 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:22.075051    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:24.329245 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:24.329214    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:24.507645 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:24.507606    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:25.124230 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.124194    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:25.361830 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.361794    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"]
Oct 30 18:38:25.362823 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.362792    5820 cpu_manager.go:414] "containerMap: removing container" podUID="49bd3da2-e0b8-4dd9-a904-626b88b3afdf" containerName="pull"
Oct 30 18:38:25.362823 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.362822    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49bd3da2-e0b8-4dd9-a904-626b88b3afdf" containerName="pull"
Oct 30 18:38:25.362993 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.362839    5820 cpu_manager.go:414] "containerMap: removing container" podUID="49bd3da2-e0b8-4dd9-a904-626b88b3afdf" containerName="extract"
Oct 30 18:38:25.362993 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.362847    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49bd3da2-e0b8-4dd9-a904-626b88b3afdf" containerName="extract"
Oct 30 18:38:25.362993 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.362868    5820 cpu_manager.go:414] "containerMap: removing container" podUID="6c5b13d2-ee83-43ef-95a9-880efafa8eec" containerName="util"
Oct 30 18:38:25.362993 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.362874    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c5b13d2-ee83-43ef-95a9-880efafa8eec" containerName="util"
Oct 30 18:38:25.362993 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.362888    5820 cpu_manager.go:414] "containerMap: removing container" podUID="6c5b13d2-ee83-43ef-95a9-880efafa8eec" containerName="extract"
Oct 30 18:38:25.362993 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.362895    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c5b13d2-ee83-43ef-95a9-880efafa8eec" containerName="extract"
Oct 30 18:38:25.362993 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.362920    5820 cpu_manager.go:414] "containerMap: removing container" podUID="6c5b13d2-ee83-43ef-95a9-880efafa8eec" containerName="pull"
Oct 30 18:38:25.362993 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.362927    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c5b13d2-ee83-43ef-95a9-880efafa8eec" containerName="pull"
Oct 30 18:38:25.362993 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.362935    5820 cpu_manager.go:414] "containerMap: removing container" podUID="49bd3da2-e0b8-4dd9-a904-626b88b3afdf" containerName="util"
Oct 30 18:38:25.362993 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.362942    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49bd3da2-e0b8-4dd9-a904-626b88b3afdf" containerName="util"
Oct 30 18:38:25.363409 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.363039    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="49bd3da2-e0b8-4dd9-a904-626b88b3afdf" containerName="extract"
Oct 30 18:38:25.363409 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.363051    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c5b13d2-ee83-43ef-95a9-880efafa8eec" containerName="extract"
Oct 30 18:38:25.365236 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.365216    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"
Oct 30 18:38:25.417936 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.417899    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 18:38:25.418191 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.418172    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 18:38:25.418637 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.418613    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 30 18:38:25.418722 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.418678    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 30 18:38:25.464589 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.464558    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:25.467726 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.467701    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 30 18:38:25.467850 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.467740    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-xt8v5\""
Oct 30 18:38:25.496152 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.496127    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"]
Oct 30 18:38:25.511318 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.511290    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/520bd39f-3402-48e2-b4b0-86c10d9ad854-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-46mfh\" (UID: \"520bd39f-3402-48e2-b4b0-86c10d9ad854\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"
Oct 30 18:38:25.511448 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.511327    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/520bd39f-3402-48e2-b4b0-86c10d9ad854-tmp\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-46mfh\" (UID: \"520bd39f-3402-48e2-b4b0-86c10d9ad854\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"
Oct 30 18:38:25.511448 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.511346    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jqmw\" (UniqueName: \"kubernetes.io/projected/520bd39f-3402-48e2-b4b0-86c10d9ad854-kube-api-access-4jqmw\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-46mfh\" (UID: \"520bd39f-3402-48e2-b4b0-86c10d9ad854\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"
Oct 30 18:38:25.511448 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.511422    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/520bd39f-3402-48e2-b4b0-86c10d9ad854-manifests\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-46mfh\" (UID: \"520bd39f-3402-48e2-b4b0-86c10d9ad854\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"
Oct 30 18:38:25.511618 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.511521    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/520bd39f-3402-48e2-b4b0-86c10d9ad854-metrics-cert\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-46mfh\" (UID: \"520bd39f-3402-48e2-b4b0-86c10d9ad854\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"
Oct 30 18:38:25.511679 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.511629    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/520bd39f-3402-48e2-b4b0-86c10d9ad854-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-46mfh\" (UID: \"520bd39f-3402-48e2-b4b0-86c10d9ad854\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"
Oct 30 18:38:25.612676 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.612647    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/520bd39f-3402-48e2-b4b0-86c10d9ad854-metrics-cert\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-46mfh\" (UID: \"520bd39f-3402-48e2-b4b0-86c10d9ad854\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"
Oct 30 18:38:25.612847 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.612698    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/520bd39f-3402-48e2-b4b0-86c10d9ad854-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-46mfh\" (UID: \"520bd39f-3402-48e2-b4b0-86c10d9ad854\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"
Oct 30 18:38:25.612847 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.612822    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/520bd39f-3402-48e2-b4b0-86c10d9ad854-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-46mfh\" (UID: \"520bd39f-3402-48e2-b4b0-86c10d9ad854\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"
Oct 30 18:38:25.612989 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.612855    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/520bd39f-3402-48e2-b4b0-86c10d9ad854-tmp\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-46mfh\" (UID: \"520bd39f-3402-48e2-b4b0-86c10d9ad854\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"
Oct 30 18:38:25.612989 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.612875    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jqmw\" (UniqueName: \"kubernetes.io/projected/520bd39f-3402-48e2-b4b0-86c10d9ad854-kube-api-access-4jqmw\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-46mfh\" (UID: \"520bd39f-3402-48e2-b4b0-86c10d9ad854\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"
Oct 30 18:38:25.612989 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.612897    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/520bd39f-3402-48e2-b4b0-86c10d9ad854-manifests\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-46mfh\" (UID: \"520bd39f-3402-48e2-b4b0-86c10d9ad854\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"
Oct 30 18:38:25.613453 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.613386    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/520bd39f-3402-48e2-b4b0-86c10d9ad854-manifests\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-46mfh\" (UID: \"520bd39f-3402-48e2-b4b0-86c10d9ad854\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"
Oct 30 18:38:25.615181 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.615150    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/520bd39f-3402-48e2-b4b0-86c10d9ad854-tmp\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-46mfh\" (UID: \"520bd39f-3402-48e2-b4b0-86c10d9ad854\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"
Oct 30 18:38:25.615380 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.615363    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/520bd39f-3402-48e2-b4b0-86c10d9ad854-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-46mfh\" (UID: \"520bd39f-3402-48e2-b4b0-86c10d9ad854\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"
Oct 30 18:38:25.615380 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.615371    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/520bd39f-3402-48e2-b4b0-86c10d9ad854-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-46mfh\" (UID: \"520bd39f-3402-48e2-b4b0-86c10d9ad854\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"
Oct 30 18:38:25.615497 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.615436    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/520bd39f-3402-48e2-b4b0-86c10d9ad854-metrics-cert\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-46mfh\" (UID: \"520bd39f-3402-48e2-b4b0-86c10d9ad854\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"
Oct 30 18:38:25.675772 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.675709    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jqmw\" (UniqueName: \"kubernetes.io/projected/520bd39f-3402-48e2-b4b0-86c10d9ad854-kube-api-access-4jqmw\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-46mfh\" (UID: \"520bd39f-3402-48e2-b4b0-86c10d9ad854\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"
Oct 30 18:38:25.681511 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.681492    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"
Oct 30 18:38:25.713479 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.713450    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:25.895365 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:25.895325    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:26.194405 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:26.194371    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"]
Oct 30 18:38:26.197382 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:38:26.197355    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod520bd39f_3402_48e2_b4b0_86c10d9ad854.slice/crio-d03e2d926fe6f760963681cb5267f0c4f06319f55a3a74233a2e239acd491f19 WatchSource:0}: Error finding container d03e2d926fe6f760963681cb5267f0c4f06319f55a3a74233a2e239acd491f19: Status 404 returned error can't find the container with id d03e2d926fe6f760963681cb5267f0c4f06319f55a3a74233a2e239acd491f19
Oct 30 18:38:26.425428 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:26.425395    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh" event={"ID":"520bd39f-3402-48e2-b4b0-86c10d9ad854","Type":"ContainerStarted","Data":"d03e2d926fe6f760963681cb5267f0c4f06319f55a3a74233a2e239acd491f19"}
Oct 30 18:38:26.519824 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:26.519719    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:28.184916 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:28.184877    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:28.429344 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:28.429307    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:30.446677 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:30.446641    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh" event={"ID":"520bd39f-3402-48e2-b4b0-86c10d9ad854","Type":"ContainerStarted","Data":"4e65e8639b44b44fbb90312827fef86d3ff41fca9aa6921d30c0b84b5f4207ad"}
Oct 30 18:38:31.451145 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:31.451108    5820 generic.go:356] "Generic (PLEG): container finished" podID="520bd39f-3402-48e2-b4b0-86c10d9ad854" containerID="4e65e8639b44b44fbb90312827fef86d3ff41fca9aa6921d30c0b84b5f4207ad" exitCode=0
Oct 30 18:38:31.451553 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:31.451189    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh" event={"ID":"520bd39f-3402-48e2-b4b0-86c10d9ad854","Type":"ContainerDied","Data":"4e65e8639b44b44fbb90312827fef86d3ff41fca9aa6921d30c0b84b5f4207ad"}
Oct 30 18:38:31.848748 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:31.848711    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:32.459218 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:32.459187    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh" event={"ID":"520bd39f-3402-48e2-b4b0-86c10d9ad854","Type":"ContainerStarted","Data":"fd8bad6a794ec5b39ff5b747431b687c22f19feca4ff730872c44fb92aeb0639"}
Oct 30 18:38:32.459604 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:32.459295    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"
Oct 30 18:38:32.705849 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:32.705758    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh" podStartSLOduration=2.21333532 podStartE2EDuration="7.705743062s" totalImagesPullingTime="5.035207963s" totalInitContainerRuntime="457.199779ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:32.656075476 +0000 UTC m=+233.367824172" watchObservedRunningTime="2026-10-30 18:38:32.705743062 +0000 UTC m=+233.417491758"
Oct 30 18:38:33.584001 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:33.583959    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:33.815187 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:33.815155    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:34.001347 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:34.001267    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:36.079311 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:36.079277    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:36.308165 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:36.308126    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:37.562712 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:37.562676    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:37.921119 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:37.921040    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:38.156573 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:38.156531    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:38.202794 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:38:38.202717    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:38:38.775497 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:38.775452    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:39.012344 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:39.012309    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:41.359737 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:41.359684    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:41.769773 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:41.769683    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:42.010588 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:42.010551    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:43.465212 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:43.465187    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-46mfh"
Oct 30 18:38:43.732453 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:43.732380    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:44.033867 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:44.033764    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:44.345785 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:44.345758    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:47.509227 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:47.509187    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:47.781315 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:47.781221    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:48.333202 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:48.333167    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:48.930781 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:48.930748    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:52.591239 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:52.591193    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:52.822601 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:52.822517    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:55.175238 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:55.175198    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:55.365004 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:55.364963    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:58.707492 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:58.707457    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:58.899886 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:58.899846    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:59.284315 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:38:59.284280    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:00.739656 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:00.739623    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:00.975558 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:00.975524    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:01.013205 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:01.013134    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-9468cdd5d-zg5gs"]
Oct 30 18:39:01.015553 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:01.015536    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-9468cdd5d-zg5gs"
Oct 30 18:39:01.113231 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:01.113195    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4de42b07-6abb-419a-99dc-89b226a2fa4e-tmp\") pod \"kube-auth-proxy-9468cdd5d-zg5gs\" (UID: \"4de42b07-6abb-419a-99dc-89b226a2fa4e\") " pod="openshift-ingress/kube-auth-proxy-9468cdd5d-zg5gs"
Oct 30 18:39:01.113231 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:01.113233    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4de42b07-6abb-419a-99dc-89b226a2fa4e-tls-certs\") pod \"kube-auth-proxy-9468cdd5d-zg5gs\" (UID: \"4de42b07-6abb-419a-99dc-89b226a2fa4e\") " pod="openshift-ingress/kube-auth-proxy-9468cdd5d-zg5gs"
Oct 30 18:39:01.113444 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:01.113253    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9drt6\" (UniqueName: \"kubernetes.io/projected/4de42b07-6abb-419a-99dc-89b226a2fa4e-kube-api-access-9drt6\") pod \"kube-auth-proxy-9468cdd5d-zg5gs\" (UID: \"4de42b07-6abb-419a-99dc-89b226a2fa4e\") " pod="openshift-ingress/kube-auth-proxy-9468cdd5d-zg5gs"
Oct 30 18:39:01.124409 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:01.124369    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 30 18:39:01.124409 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:01.124394    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8m6s4\""
Oct 30 18:39:01.124409 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:01.124398    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 30 18:39:01.125317 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:01.125298    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-9468cdd5d-zg5gs"]
Oct 30 18:39:01.167351 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:01.167312    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:01.213924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:01.213893    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4de42b07-6abb-419a-99dc-89b226a2fa4e-tls-certs\") pod \"kube-auth-proxy-9468cdd5d-zg5gs\" (UID: \"4de42b07-6abb-419a-99dc-89b226a2fa4e\") " pod="openshift-ingress/kube-auth-proxy-9468cdd5d-zg5gs"
Oct 30 18:39:01.213924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:01.213925    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9drt6\" (UniqueName: \"kubernetes.io/projected/4de42b07-6abb-419a-99dc-89b226a2fa4e-kube-api-access-9drt6\") pod \"kube-auth-proxy-9468cdd5d-zg5gs\" (UID: \"4de42b07-6abb-419a-99dc-89b226a2fa4e\") " pod="openshift-ingress/kube-auth-proxy-9468cdd5d-zg5gs"
Oct 30 18:39:01.214183 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:01.214023    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4de42b07-6abb-419a-99dc-89b226a2fa4e-tmp\") pod \"kube-auth-proxy-9468cdd5d-zg5gs\" (UID: \"4de42b07-6abb-419a-99dc-89b226a2fa4e\") " pod="openshift-ingress/kube-auth-proxy-9468cdd5d-zg5gs"
Oct 30 18:39:01.216583 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:01.216525    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4de42b07-6abb-419a-99dc-89b226a2fa4e-tmp\") pod \"kube-auth-proxy-9468cdd5d-zg5gs\" (UID: \"4de42b07-6abb-419a-99dc-89b226a2fa4e\") " pod="openshift-ingress/kube-auth-proxy-9468cdd5d-zg5gs"
Oct 30 18:39:01.216583 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:01.216577    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4de42b07-6abb-419a-99dc-89b226a2fa4e-tls-certs\") pod \"kube-auth-proxy-9468cdd5d-zg5gs\" (UID: \"4de42b07-6abb-419a-99dc-89b226a2fa4e\") " pod="openshift-ingress/kube-auth-proxy-9468cdd5d-zg5gs"
Oct 30 18:39:01.274931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:01.274862    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9drt6\" (UniqueName: \"kubernetes.io/projected/4de42b07-6abb-419a-99dc-89b226a2fa4e-kube-api-access-9drt6\") pod \"kube-auth-proxy-9468cdd5d-zg5gs\" (UID: \"4de42b07-6abb-419a-99dc-89b226a2fa4e\") " pod="openshift-ingress/kube-auth-proxy-9468cdd5d-zg5gs"
Oct 30 18:39:01.326639 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:01.326608    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-9468cdd5d-zg5gs"
Oct 30 18:39:01.350069 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:01.350044    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:01.546115 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:01.546034    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:01.717643 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:01.717608    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-9468cdd5d-zg5gs"]
Oct 30 18:39:01.722730 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:39:01.722704    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4de42b07_6abb_419a_99dc_89b226a2fa4e.slice/crio-859234c80a10a020573499d98a8bf3dd13b6de7385441e1b248eaf31134cb45e WatchSource:0}: Error finding container 859234c80a10a020573499d98a8bf3dd13b6de7385441e1b248eaf31134cb45e: Status 404 returned error can't find the container with id 859234c80a10a020573499d98a8bf3dd13b6de7385441e1b248eaf31134cb45e
Oct 30 18:39:01.778092 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:01.778056    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:01.985735 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:01.985700    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:02.579782 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:02.579749    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-9468cdd5d-zg5gs" event={"ID":"4de42b07-6abb-419a-99dc-89b226a2fa4e","Type":"ContainerStarted","Data":"859234c80a10a020573499d98a8bf3dd13b6de7385441e1b248eaf31134cb45e"}
Oct 30 18:39:03.494466 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:03.494430    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:05.595369 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:05.595338    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-9468cdd5d-zg5gs" event={"ID":"4de42b07-6abb-419a-99dc-89b226a2fa4e","Type":"ContainerStarted","Data":"20579dc0f1d99f441b29302888dedffa62159dc43166bf0a6c29aabffe61a675"}
Oct 30 18:39:05.715605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:05.715557    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-9468cdd5d-zg5gs" podStartSLOduration=2.757312436 podStartE2EDuration="5.71554207s" totalImagesPullingTime="2.958229634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:39:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:39:05.713907732 +0000 UTC m=+266.425656427" watchObservedRunningTime="2026-10-30 18:39:05.71554207 +0000 UTC m=+266.427290758"
Oct 30 18:39:07.784185 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:07.784154    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:07.963815 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:07.963779    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:08.140926 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:08.140871    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:08.202753 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:39:08.202718    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:39:08.720287 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:08.720240    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:08.781771 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:08.781739    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-w8wvn"]
Oct 30 18:39:08.785255 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:08.785234    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-w8wvn"
Oct 30 18:39:08.839001 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:08.838969    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4m2vx\""
Oct 30 18:39:08.874484 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:08.874456    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 18:39:08.888537 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:08.888507    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 18:39:08.888736 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:08.888717    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5-config-file\") pod \"limitador-limitador-74dfcfb894-w8wvn\" (UID: \"3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-w8wvn"
Oct 30 18:39:08.888796 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:08.888755    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnlzz\" (UniqueName: \"kubernetes.io/projected/3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5-kube-api-access-mnlzz\") pod \"limitador-limitador-74dfcfb894-w8wvn\" (UID: \"3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-w8wvn"
Oct 30 18:39:08.888858 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:08.888830    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 18:39:08.889750 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:08.889733    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-w8wvn"]
Oct 30 18:39:08.905540 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:08.905515    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:08.989906 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:08.989815    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5-config-file\") pod \"limitador-limitador-74dfcfb894-w8wvn\" (UID: \"3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-w8wvn"
Oct 30 18:39:08.989906 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:08.989857    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnlzz\" (UniqueName: \"kubernetes.io/projected/3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5-kube-api-access-mnlzz\") pod \"limitador-limitador-74dfcfb894-w8wvn\" (UID: \"3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-w8wvn"
Oct 30 18:39:08.990446 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:08.990428    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5-config-file\") pod \"limitador-limitador-74dfcfb894-w8wvn\" (UID: \"3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-w8wvn"
Oct 30 18:39:09.049577 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:09.049543    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-w8wvn"]
Oct 30 18:39:09.075067 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:09.075042    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnlzz\" (UniqueName: \"kubernetes.io/projected/3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5-kube-api-access-mnlzz\") pod \"limitador-limitador-74dfcfb894-w8wvn\" (UID: \"3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-w8wvn"
Oct 30 18:39:09.097824 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:09.097802    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-w8wvn"
Oct 30 18:39:09.110320 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:09.110295    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:09.304664 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:09.304631    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:09.479567 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:09.479405    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-w8wvn"]
Oct 30 18:39:09.482240 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:39:09.482208    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c5f5cd9_814a_49af_ba4f_9a5dfe28a6c5.slice/crio-ca2e21acd44daa453802c00f94b3a24cbef37dd04c8362995495b6ba4eecfd9c WatchSource:0}: Error finding container ca2e21acd44daa453802c00f94b3a24cbef37dd04c8362995495b6ba4eecfd9c: Status 404 returned error can't find the container with id ca2e21acd44daa453802c00f94b3a24cbef37dd04c8362995495b6ba4eecfd9c
Oct 30 18:39:09.612107 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:09.612034    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-w8wvn" event={"ID":"3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5","Type":"ContainerStarted","Data":"ca2e21acd44daa453802c00f94b3a24cbef37dd04c8362995495b6ba4eecfd9c"}
Oct 30 18:39:10.517184 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:10.517152    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:10.706033 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:10.705995    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:12.500759 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:12.500725    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:12.698174 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:12.698141    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:13.413918 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.413879    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-5w6kz"]
Oct 30 18:39:13.421547 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.421513    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-5w6kz"
Oct 30 18:39:13.429599 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.429567    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksglr\" (UniqueName: \"kubernetes.io/projected/9eaa27bd-09dc-46f4-acee-652fc47950fb-kube-api-access-ksglr\") pod \"authorino-56c6c9ddd5-5w6kz\" (UID: \"9eaa27bd-09dc-46f4-acee-652fc47950fb\") " pod="kuadrant-system/authorino-56c6c9ddd5-5w6kz"
Oct 30 18:39:13.477610 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.477581    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-5w6kz"]
Oct 30 18:39:13.526161 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.526137    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-fl2x7\""
Oct 30 18:39:13.530622 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.530594    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksglr\" (UniqueName: \"kubernetes.io/projected/9eaa27bd-09dc-46f4-acee-652fc47950fb-kube-api-access-ksglr\") pod \"authorino-56c6c9ddd5-5w6kz\" (UID: \"9eaa27bd-09dc-46f4-acee-652fc47950fb\") " pod="kuadrant-system/authorino-56c6c9ddd5-5w6kz"
Oct 30 18:39:13.535405 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.535379    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-dtgl9"]
Oct 30 18:39:13.538695 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.538679    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-dtgl9"
Oct 30 18:39:13.592804 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.592780    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksglr\" (UniqueName: \"kubernetes.io/projected/9eaa27bd-09dc-46f4-acee-652fc47950fb-kube-api-access-ksglr\") pod \"authorino-56c6c9ddd5-5w6kz\" (UID: \"9eaa27bd-09dc-46f4-acee-652fc47950fb\") " pod="kuadrant-system/authorino-56c6c9ddd5-5w6kz"
Oct 30 18:39:13.597113 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.597089    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-dtgl9"]
Oct 30 18:39:13.605300 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.605280    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 18:39:13.605404 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.605324    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 18:39:13.631347 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.631324    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76f5d060-4e95-48d3-99e3-815bac778442-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-dtgl9\" (UID: \"76f5d060-4e95-48d3-99e3-815bac778442\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-dtgl9"
Oct 30 18:39:13.631482 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.631378    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j42n4\" (UniqueName: \"kubernetes.io/projected/76f5d060-4e95-48d3-99e3-815bac778442-kube-api-access-j42n4\") pod \"kserve-module-controller-manager-7566cf5fc8-dtgl9\" (UID: \"76f5d060-4e95-48d3-99e3-815bac778442\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-dtgl9"
Oct 30 18:39:13.631482 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.631469    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/76f5d060-4e95-48d3-99e3-815bac778442-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-dtgl9\" (UID: \"76f5d060-4e95-48d3-99e3-815bac778442\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-dtgl9"
Oct 30 18:39:13.631583 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.631511    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/76f5d060-4e95-48d3-99e3-815bac778442-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-dtgl9\" (UID: \"76f5d060-4e95-48d3-99e3-815bac778442\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-dtgl9"
Oct 30 18:39:13.633014 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.632992    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-w8wvn" event={"ID":"3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5","Type":"ContainerStarted","Data":"7bca9e88592feaf8c0eeebbe61da1196b550673d3a61710bd4626f26f0c06a8e"}
Oct 30 18:39:13.633166 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.633147    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-w8wvn"
Oct 30 18:39:13.695037 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.694982    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Oct 30 18:39:13.704508 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.704487    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-7d7ww\""
Oct 30 18:39:13.732479 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.732450    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/76f5d060-4e95-48d3-99e3-815bac778442-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-dtgl9\" (UID: \"76f5d060-4e95-48d3-99e3-815bac778442\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-dtgl9"
Oct 30 18:39:13.732588 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.732538    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76f5d060-4e95-48d3-99e3-815bac778442-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-dtgl9\" (UID: \"76f5d060-4e95-48d3-99e3-815bac778442\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-dtgl9"
Oct 30 18:39:13.732651 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.732618    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j42n4\" (UniqueName: \"kubernetes.io/projected/76f5d060-4e95-48d3-99e3-815bac778442-kube-api-access-j42n4\") pod \"kserve-module-controller-manager-7566cf5fc8-dtgl9\" (UID: \"76f5d060-4e95-48d3-99e3-815bac778442\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-dtgl9"
Oct 30 18:39:13.732707 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.732656    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/76f5d060-4e95-48d3-99e3-815bac778442-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-dtgl9\" (UID: \"76f5d060-4e95-48d3-99e3-815bac778442\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-dtgl9"
Oct 30 18:39:13.732999 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.732980    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76f5d060-4e95-48d3-99e3-815bac778442-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-dtgl9\" (UID: \"76f5d060-4e95-48d3-99e3-815bac778442\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-dtgl9"
Oct 30 18:39:13.733075 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.733002    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/76f5d060-4e95-48d3-99e3-815bac778442-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-dtgl9\" (UID: \"76f5d060-4e95-48d3-99e3-815bac778442\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-dtgl9"
Oct 30 18:39:13.733381 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.733362    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/76f5d060-4e95-48d3-99e3-815bac778442-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-dtgl9\" (UID: \"76f5d060-4e95-48d3-99e3-815bac778442\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-dtgl9"
Oct 30 18:39:13.748030 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.748010    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-5w6kz"
Oct 30 18:39:13.795334 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.794493    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j42n4\" (UniqueName: \"kubernetes.io/projected/76f5d060-4e95-48d3-99e3-815bac778442-kube-api-access-j42n4\") pod \"kserve-module-controller-manager-7566cf5fc8-dtgl9\" (UID: \"76f5d060-4e95-48d3-99e3-815bac778442\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-dtgl9"
Oct 30 18:39:13.820982 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.820621    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-w8wvn" podStartSLOduration=2.363690526 podStartE2EDuration="5.820606317s" totalImagesPullingTime="3.456915791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:39:13.819565717 +0000 UTC m=+274.531314415" watchObservedRunningTime="2026-10-30 18:39:13.820606317 +0000 UTC m=+274.532355005"
Oct 30 18:39:13.851095 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:13.851064    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-dtgl9"
Oct 30 18:39:14.117991 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:14.117972    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-5w6kz"]
Oct 30 18:39:14.120391 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:39:14.120367    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9eaa27bd_09dc_46f4_acee_652fc47950fb.slice/crio-8dae66552ca251e13b4a7b64288408443df0901805d5f6e0249afe178126842d WatchSource:0}: Error finding container 8dae66552ca251e13b4a7b64288408443df0901805d5f6e0249afe178126842d: Status 404 returned error can't find the container with id 8dae66552ca251e13b4a7b64288408443df0901805d5f6e0249afe178126842d
Oct 30 18:39:14.229909 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:14.229883    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-dtgl9"]
Oct 30 18:39:14.232415 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:39:14.232389    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76f5d060_4e95_48d3_99e3_815bac778442.slice/crio-55b9d5809e2c6dfda7bf897d9abbb945ef7ed7a99c045986449d314d296a135f WatchSource:0}: Error finding container 55b9d5809e2c6dfda7bf897d9abbb945ef7ed7a99c045986449d314d296a135f: Status 404 returned error can't find the container with id 55b9d5809e2c6dfda7bf897d9abbb945ef7ed7a99c045986449d314d296a135f
Oct 30 18:39:14.391036 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:14.390919    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:14.583078 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:14.583045    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:14.638679 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:14.638647    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-5w6kz" event={"ID":"9eaa27bd-09dc-46f4-acee-652fc47950fb","Type":"ContainerStarted","Data":"8dae66552ca251e13b4a7b64288408443df0901805d5f6e0249afe178126842d"}
Oct 30 18:39:14.639744 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:14.639714    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-dtgl9" event={"ID":"76f5d060-4e95-48d3-99e3-815bac778442","Type":"ContainerStarted","Data":"55b9d5809e2c6dfda7bf897d9abbb945ef7ed7a99c045986449d314d296a135f"}
Oct 30 18:39:16.650693 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:16.650657    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-dtgl9" event={"ID":"76f5d060-4e95-48d3-99e3-815bac778442","Type":"ContainerStarted","Data":"8a8bbaa4a196a8ab9223769ee842e79e94d2cc93e41546f9033a97e82c2a80cf"}
Oct 30 18:39:16.651151 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:16.650734    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-dtgl9"
Oct 30 18:39:16.773836 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:16.773786    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-dtgl9" podStartSLOduration=1.879116025 podStartE2EDuration="3.773769198s" totalImagesPullingTime="1.894653173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:39:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:39:16.772600884 +0000 UTC m=+277.484349580" watchObservedRunningTime="2026-10-30 18:39:16.773769198 +0000 UTC m=+277.485517896"
Oct 30 18:39:17.661646 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:17.661563    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-5w6kz" event={"ID":"9eaa27bd-09dc-46f4-acee-652fc47950fb","Type":"ContainerStarted","Data":"7ac1d826aba8ab8dfc2fd8f12cebe0d8c2c33e72447cb93c738c88d9544eba40"}
Oct 30 18:39:17.776322 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:17.776266    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-5w6kz" podStartSLOduration=2.199387055 podStartE2EDuration="4.776243614s" totalImagesPullingTime="2.576856559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:39:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:39:17.775101413 +0000 UTC m=+278.486850110" watchObservedRunningTime="2026-10-30 18:39:17.776243614 +0000 UTC m=+278.487992313"
Oct 30 18:39:19.890664 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:19.890627    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:20.118851 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:20.118816    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:20.311032 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:20.310998    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:23.587799 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:23.587752    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:23.760585 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:23.760549    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:24.641235 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:24.641208    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-w8wvn"
Oct 30 18:39:27.424156 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:27.424124    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:27.649040 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:27.649004    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:28.148602 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:28.148557    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:28.326002 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:28.325968    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:28.654661 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:28.654627    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-w8wvn"]
Oct 30 18:39:28.655100 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:28.654804    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-w8wvn" podUID="3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5" containerName="limitador" containerID="cri-o://7bca9e88592feaf8c0eeebbe61da1196b550673d3a61710bd4626f26f0c06a8e" gracePeriod=30
Oct 30 18:39:29.191029 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:29.191007    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-w8wvn"
Oct 30 18:39:29.267715 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:29.267641    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5-kube-api-access-mnlzz\" (UniqueName: \"kubernetes.io/projected/3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5-kube-api-access-mnlzz\") pod \"3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5\" (UID: \"3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5\") "
Oct 30 18:39:29.267715 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:29.267688    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5-config-file\" (UniqueName: \"kubernetes.io/configmap/3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5-config-file\") pod \"3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5\" (UID: \"3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5\") "
Oct 30 18:39:29.268118 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:29.268096    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5-config-file" pod "3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5" (UID: "3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:39:29.269988 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:29.269966    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5-kube-api-access-mnlzz" pod "3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5" (UID: "3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5"). InnerVolumeSpecName "kube-api-access-mnlzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:39:29.368291 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:29.368249    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnlzz\" (UniqueName: \"kubernetes.io/projected/3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5-kube-api-access-mnlzz\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:39:29.368291 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:29.368284    5820 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5-config-file\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:39:29.711044 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:29.711003    5820 generic.go:356] "Generic (PLEG): container finished" podID="3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5" containerID="7bca9e88592feaf8c0eeebbe61da1196b550673d3a61710bd4626f26f0c06a8e" exitCode=0
Oct 30 18:39:29.711472 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:29.711058    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-w8wvn"
Oct 30 18:39:29.711472 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:29.711065    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-w8wvn" event={"ID":"3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5","Type":"ContainerDied","Data":"7bca9e88592feaf8c0eeebbe61da1196b550673d3a61710bd4626f26f0c06a8e"}
Oct 30 18:39:29.711472 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:29.711118    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-w8wvn" event={"ID":"3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5","Type":"ContainerDied","Data":"ca2e21acd44daa453802c00f94b3a24cbef37dd04c8362995495b6ba4eecfd9c"}
Oct 30 18:39:29.711472 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:29.711148    5820 scope.go:122] "RemoveContainer" containerID="7bca9e88592feaf8c0eeebbe61da1196b550673d3a61710bd4626f26f0c06a8e"
Oct 30 18:39:29.719469 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:29.719439    5820 scope.go:122] "RemoveContainer" containerID="7bca9e88592feaf8c0eeebbe61da1196b550673d3a61710bd4626f26f0c06a8e"
Oct 30 18:39:29.719717 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:39:29.719701    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bca9e88592feaf8c0eeebbe61da1196b550673d3a61710bd4626f26f0c06a8e\": container with ID starting with 7bca9e88592feaf8c0eeebbe61da1196b550673d3a61710bd4626f26f0c06a8e not found: ID does not exist" containerID="7bca9e88592feaf8c0eeebbe61da1196b550673d3a61710bd4626f26f0c06a8e"
Oct 30 18:39:29.719774 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:29.719725    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bca9e88592feaf8c0eeebbe61da1196b550673d3a61710bd4626f26f0c06a8e"} err="failed to get container status \"7bca9e88592feaf8c0eeebbe61da1196b550673d3a61710bd4626f26f0c06a8e\": rpc error: code = NotFound desc = could not find container \"7bca9e88592feaf8c0eeebbe61da1196b550673d3a61710bd4626f26f0c06a8e\": container with ID starting with 7bca9e88592feaf8c0eeebbe61da1196b550673d3a61710bd4626f26f0c06a8e not found: ID does not exist"
Oct 30 18:39:29.884323 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:29.884284    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-w8wvn"]
Oct 30 18:39:29.933513 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:29.933485    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-w8wvn"]
Oct 30 18:39:31.435305 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:31.435263    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5" path="/var/lib/kubelet/pods/3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5/volumes"
Oct 30 18:39:33.787742 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:33.787699    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:33.966164 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:33.966132    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:36.682960 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:36.682910    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:36.874368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:36.874336    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:37.527241 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:37.527204    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-j57nn"]
Oct 30 18:39:37.527545 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:37.527532    5820 cpu_manager.go:414] "containerMap: removing container" podUID="3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5" containerName="limitador"
Oct 30 18:39:37.527589 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:37.527546    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5" containerName="limitador"
Oct 30 18:39:37.527627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:37.527617    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c5f5cd9-814a-49af-ba4f-9a5dfe28a6c5" containerName="limitador"
Oct 30 18:39:37.531809 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:37.531790    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-j57nn"
Oct 30 18:39:37.585381 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:37.585354    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 18:39:37.585554 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:37.585448    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 18:39:37.635218 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:37.635193    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-vmnz4\""
Oct 30 18:39:37.635370 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:37.635192    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 30 18:39:37.636176 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:37.636160    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-j57nn"]
Oct 30 18:39:37.640662 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:37.640645    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6jjd\" (UniqueName: \"kubernetes.io/projected/4bab49b9-aab3-4d20-abc5-ed06cd6964fa-kube-api-access-c6jjd\") pod \"postgres-7cdbf6dc44-j57nn\" (UID: \"4bab49b9-aab3-4d20-abc5-ed06cd6964fa\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-j57nn"
Oct 30 18:39:37.640713 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:37.640679    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4bab49b9-aab3-4d20-abc5-ed06cd6964fa-data\") pod \"postgres-7cdbf6dc44-j57nn\" (UID: \"4bab49b9-aab3-4d20-abc5-ed06cd6964fa\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-j57nn"
Oct 30 18:39:37.663058 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:37.663035    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-dtgl9"
Oct 30 18:39:37.669626 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:37.669604    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:37.741666 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:37.741631    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6jjd\" (UniqueName: \"kubernetes.io/projected/4bab49b9-aab3-4d20-abc5-ed06cd6964fa-kube-api-access-c6jjd\") pod \"postgres-7cdbf6dc44-j57nn\" (UID: \"4bab49b9-aab3-4d20-abc5-ed06cd6964fa\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-j57nn"
Oct 30 18:39:37.742076 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:37.741695    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4bab49b9-aab3-4d20-abc5-ed06cd6964fa-data\") pod \"postgres-7cdbf6dc44-j57nn\" (UID: \"4bab49b9-aab3-4d20-abc5-ed06cd6964fa\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-j57nn"
Oct 30 18:39:37.742232 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:37.742212    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4bab49b9-aab3-4d20-abc5-ed06cd6964fa-data\") pod \"postgres-7cdbf6dc44-j57nn\" (UID: \"4bab49b9-aab3-4d20-abc5-ed06cd6964fa\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-j57nn"
Oct 30 18:39:37.804181 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:37.804160    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6jjd\" (UniqueName: \"kubernetes.io/projected/4bab49b9-aab3-4d20-abc5-ed06cd6964fa-kube-api-access-c6jjd\") pod \"postgres-7cdbf6dc44-j57nn\" (UID: \"4bab49b9-aab3-4d20-abc5-ed06cd6964fa\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-j57nn"
Oct 30 18:39:37.843386 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:37.843362    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-j57nn"
Oct 30 18:39:38.202630 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:39:38.202548    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:39:38.219661 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:38.219635    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-j57nn"]
Oct 30 18:39:38.223269 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:39:38.223236    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bab49b9_aab3_4d20_abc5_ed06cd6964fa.slice/crio-2e45e7fb0e88edce7cc2c0460957830df654cb56dfe9066d150a743aca1be39f WatchSource:0}: Error finding container 2e45e7fb0e88edce7cc2c0460957830df654cb56dfe9066d150a743aca1be39f: Status 404 returned error can't find the container with id 2e45e7fb0e88edce7cc2c0460957830df654cb56dfe9066d150a743aca1be39f
Oct 30 18:39:38.750570 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:38.750534    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-j57nn" event={"ID":"4bab49b9-aab3-4d20-abc5-ed06cd6964fa","Type":"ContainerStarted","Data":"2e45e7fb0e88edce7cc2c0460957830df654cb56dfe9066d150a743aca1be39f"}
Oct 30 18:39:44.635373 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:44.635341    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 30 18:39:44.779832 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:44.779795    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-j57nn" event={"ID":"4bab49b9-aab3-4d20-abc5-ed06cd6964fa","Type":"ContainerStarted","Data":"8c64f7d9e2ed76410afcc6842d2623fdfc8eb6e61ab162db51a61920ad2cc65a"}
Oct 30 18:39:44.780053 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:44.779939    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-j57nn"
Oct 30 18:39:44.896528 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:44.896427    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-j57nn" podStartSLOduration=1.537663201 podStartE2EDuration="7.896413573s" totalImagesPullingTime="6.358750372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:39:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:39:44.895278138 +0000 UTC m=+305.607026835" watchObservedRunningTime="2026-10-30 18:39:44.896413573 +0000 UTC m=+305.608162307"
Oct 30 18:39:47.905882 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:47.905848    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:50.813329 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:50.813301    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-j57nn"
Oct 30 18:39:58.326339 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:58.326304    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:58.523524 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:58.523474    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:58.699251 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:58.699159    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:58.884729 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:39:58.884688    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:06.004348 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:06.004310    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:06.189876 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:06.189840    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:08.202502 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:40:08.202464    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:40:08.988089 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:08.988060    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw"]
Oct 30 18:40:08.995142 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:08.995120    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw"
Oct 30 18:40:09.031640 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:09.031609    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d0e5ae11-c274-4e34-a6fe-ac8df2700247-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mcgqw\" (UID: \"d0e5ae11-c274-4e34-a6fe-ac8df2700247\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw"
Oct 30 18:40:09.031796 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:09.031704    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0e5ae11-c274-4e34-a6fe-ac8df2700247-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mcgqw\" (UID: \"d0e5ae11-c274-4e34-a6fe-ac8df2700247\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw"
Oct 30 18:40:09.031796 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:09.031758    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjdzl\" (UniqueName: \"kubernetes.io/projected/d0e5ae11-c274-4e34-a6fe-ac8df2700247-kube-api-access-zjdzl\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mcgqw\" (UID: \"d0e5ae11-c274-4e34-a6fe-ac8df2700247\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw"
Oct 30 18:40:09.031796 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:09.031787    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/d0e5ae11-c274-4e34-a6fe-ac8df2700247-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mcgqw\" (UID: \"d0e5ae11-c274-4e34-a6fe-ac8df2700247\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw"
Oct 30 18:40:09.048667 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:09.048638    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Oct 30 18:40:09.098313 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:09.098283    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Oct 30 18:40:09.098500 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:09.098386    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Oct 30 18:40:09.098500 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:09.098491    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-6chz2\""
Oct 30 18:40:09.099678 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:09.099658    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw"]
Oct 30 18:40:09.132533 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:09.132504    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjdzl\" (UniqueName: \"kubernetes.io/projected/d0e5ae11-c274-4e34-a6fe-ac8df2700247-kube-api-access-zjdzl\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mcgqw\" (UID: \"d0e5ae11-c274-4e34-a6fe-ac8df2700247\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw"
Oct 30 18:40:09.132703 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:09.132545    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/d0e5ae11-c274-4e34-a6fe-ac8df2700247-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mcgqw\" (UID: \"d0e5ae11-c274-4e34-a6fe-ac8df2700247\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw"
Oct 30 18:40:09.132703 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:09.132599    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d0e5ae11-c274-4e34-a6fe-ac8df2700247-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mcgqw\" (UID: \"d0e5ae11-c274-4e34-a6fe-ac8df2700247\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw"
Oct 30 18:40:09.132703 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:09.132623    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0e5ae11-c274-4e34-a6fe-ac8df2700247-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mcgqw\" (UID: \"d0e5ae11-c274-4e34-a6fe-ac8df2700247\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw"
Oct 30 18:40:09.132703 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:40:09.132696    5820 configmap.go:193] Couldn't get configMap opendatahub/usage-tenancy-proxy-source: configmap "usage-tenancy-proxy-source" not found
Oct 30 18:40:09.132902 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:40:09.132762    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d0e5ae11-c274-4e34-a6fe-ac8df2700247-proxy-source podName:d0e5ae11-c274-4e34-a6fe-ac8df2700247 nodeName:}" failed. No retries permitted until 2026-10-30 18:40:09.632743453 +0000 UTC m=+330.344492154 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-source" (UniqueName: "kubernetes.io/configmap/d0e5ae11-c274-4e34-a6fe-ac8df2700247-proxy-source") pod "usage-logs-tenancy-proxy-9c5d49bcd-mcgqw" (UID: "d0e5ae11-c274-4e34-a6fe-ac8df2700247") : configmap "usage-tenancy-proxy-source" not found
Oct 30 18:40:09.132902 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:40:09.132766    5820 secret.go:189] Couldn't get secret opendatahub/usage-logs-tenancy-proxy-tls: secret "usage-logs-tenancy-proxy-tls" not found
Oct 30 18:40:09.132902 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:40:09.132831    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0e5ae11-c274-4e34-a6fe-ac8df2700247-tls-cert podName:d0e5ae11-c274-4e34-a6fe-ac8df2700247 nodeName:}" failed. No retries permitted until 2026-10-30 18:40:09.632812607 +0000 UTC m=+330.344561300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-cert" (UniqueName: "kubernetes.io/secret/d0e5ae11-c274-4e34-a6fe-ac8df2700247-tls-cert") pod "usage-logs-tenancy-proxy-9c5d49bcd-mcgqw" (UID: "d0e5ae11-c274-4e34-a6fe-ac8df2700247") : secret "usage-logs-tenancy-proxy-tls" not found
Oct 30 18:40:09.133366 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:09.133349    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0e5ae11-c274-4e34-a6fe-ac8df2700247-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mcgqw\" (UID: \"d0e5ae11-c274-4e34-a6fe-ac8df2700247\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw"
Oct 30 18:40:09.192535 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:09.192501    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:09.194599 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:09.194571    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjdzl\" (UniqueName: \"kubernetes.io/projected/d0e5ae11-c274-4e34-a6fe-ac8df2700247-kube-api-access-zjdzl\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mcgqw\" (UID: \"d0e5ae11-c274-4e34-a6fe-ac8df2700247\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw"
Oct 30 18:40:09.417153 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:09.417116    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:09.636124 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:09.636085    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d0e5ae11-c274-4e34-a6fe-ac8df2700247-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mcgqw\" (UID: \"d0e5ae11-c274-4e34-a6fe-ac8df2700247\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw"
Oct 30 18:40:09.636304 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:09.636152    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/d0e5ae11-c274-4e34-a6fe-ac8df2700247-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mcgqw\" (UID: \"d0e5ae11-c274-4e34-a6fe-ac8df2700247\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw"
Oct 30 18:40:09.637002 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:09.636973    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/d0e5ae11-c274-4e34-a6fe-ac8df2700247-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mcgqw\" (UID: \"d0e5ae11-c274-4e34-a6fe-ac8df2700247\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw"
Oct 30 18:40:09.638712 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:09.638687    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d0e5ae11-c274-4e34-a6fe-ac8df2700247-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mcgqw\" (UID: \"d0e5ae11-c274-4e34-a6fe-ac8df2700247\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw"
Oct 30 18:40:09.908689 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:09.908657    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw"
Oct 30 18:40:10.336572 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:10.336532    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw"]
Oct 30 18:40:10.339472 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:40:10.339444    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0e5ae11_c274_4e34_a6fe_ac8df2700247.slice/crio-f3edec6fb044e7aeeda9b8dc1071008ded1cf5fc7d6fa542919a79dbe6f4a6db WatchSource:0}: Error finding container f3edec6fb044e7aeeda9b8dc1071008ded1cf5fc7d6fa542919a79dbe6f4a6db: Status 404 returned error can't find the container with id f3edec6fb044e7aeeda9b8dc1071008ded1cf5fc7d6fa542919a79dbe6f4a6db
Oct 30 18:40:10.899455 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:10.899414    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw" event={"ID":"d0e5ae11-c274-4e34-a6fe-ac8df2700247","Type":"ContainerStarted","Data":"f3edec6fb044e7aeeda9b8dc1071008ded1cf5fc7d6fa542919a79dbe6f4a6db"}
Oct 30 18:40:14.363215 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.363181    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"]
Oct 30 18:40:14.366572 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.366553    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:14.470617 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.470583    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"]
Oct 30 18:40:14.480826 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.480737    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/75209be8-a14e-40e9-9db9-d0a276c958f2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-94chw\" (UID: \"75209be8-a14e-40e9-9db9-d0a276c958f2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:14.480826 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.480784    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/75209be8-a14e-40e9-9db9-d0a276c958f2-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-94chw\" (UID: \"75209be8-a14e-40e9-9db9-d0a276c958f2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:14.481755 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.480885    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/75209be8-a14e-40e9-9db9-d0a276c958f2-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-94chw\" (UID: \"75209be8-a14e-40e9-9db9-d0a276c958f2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:14.481755 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.480922    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75209be8-a14e-40e9-9db9-d0a276c958f2-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-94chw\" (UID: \"75209be8-a14e-40e9-9db9-d0a276c958f2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:14.481755 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.481010    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/75209be8-a14e-40e9-9db9-d0a276c958f2-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-94chw\" (UID: \"75209be8-a14e-40e9-9db9-d0a276c958f2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:14.481755 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.481074    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v96l2\" (UniqueName: \"kubernetes.io/projected/75209be8-a14e-40e9-9db9-d0a276c958f2-kube-api-access-v96l2\") pod \"istiod-openshift-gateway-7ddd9cfdc8-94chw\" (UID: \"75209be8-a14e-40e9-9db9-d0a276c958f2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:14.481755 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.481123    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/75209be8-a14e-40e9-9db9-d0a276c958f2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-94chw\" (UID: \"75209be8-a14e-40e9-9db9-d0a276c958f2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:14.582562 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.582520    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/75209be8-a14e-40e9-9db9-d0a276c958f2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-94chw\" (UID: \"75209be8-a14e-40e9-9db9-d0a276c958f2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:14.582562 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.582568    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/75209be8-a14e-40e9-9db9-d0a276c958f2-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-94chw\" (UID: \"75209be8-a14e-40e9-9db9-d0a276c958f2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:14.582796 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.582588    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/75209be8-a14e-40e9-9db9-d0a276c958f2-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-94chw\" (UID: \"75209be8-a14e-40e9-9db9-d0a276c958f2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:14.582796 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.582609    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75209be8-a14e-40e9-9db9-d0a276c958f2-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-94chw\" (UID: \"75209be8-a14e-40e9-9db9-d0a276c958f2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:14.582796 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.582655    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/75209be8-a14e-40e9-9db9-d0a276c958f2-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-94chw\" (UID: \"75209be8-a14e-40e9-9db9-d0a276c958f2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:14.582796 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.582718    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v96l2\" (UniqueName: \"kubernetes.io/projected/75209be8-a14e-40e9-9db9-d0a276c958f2-kube-api-access-v96l2\") pod \"istiod-openshift-gateway-7ddd9cfdc8-94chw\" (UID: \"75209be8-a14e-40e9-9db9-d0a276c958f2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:14.582796 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.582771    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/75209be8-a14e-40e9-9db9-d0a276c958f2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-94chw\" (UID: \"75209be8-a14e-40e9-9db9-d0a276c958f2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:14.583339 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.583311    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/75209be8-a14e-40e9-9db9-d0a276c958f2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-94chw\" (UID: \"75209be8-a14e-40e9-9db9-d0a276c958f2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:14.585659 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.585623    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/75209be8-a14e-40e9-9db9-d0a276c958f2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-94chw\" (UID: \"75209be8-a14e-40e9-9db9-d0a276c958f2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:14.587224 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.587199    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/75209be8-a14e-40e9-9db9-d0a276c958f2-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-94chw\" (UID: \"75209be8-a14e-40e9-9db9-d0a276c958f2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:14.588186 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.588166    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/75209be8-a14e-40e9-9db9-d0a276c958f2-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-94chw\" (UID: \"75209be8-a14e-40e9-9db9-d0a276c958f2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:14.597426 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.597398    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/75209be8-a14e-40e9-9db9-d0a276c958f2-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-94chw\" (UID: \"75209be8-a14e-40e9-9db9-d0a276c958f2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:14.646079 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.646006    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75209be8-a14e-40e9-9db9-d0a276c958f2-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-94chw\" (UID: \"75209be8-a14e-40e9-9db9-d0a276c958f2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:14.651401 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.651374    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v96l2\" (UniqueName: \"kubernetes.io/projected/75209be8-a14e-40e9-9db9-d0a276c958f2-kube-api-access-v96l2\") pod \"istiod-openshift-gateway-7ddd9cfdc8-94chw\" (UID: \"75209be8-a14e-40e9-9db9-d0a276c958f2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:14.679295 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:14.679037    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:15.141936 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:15.141904    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"]
Oct 30 18:40:15.173650 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:40:15.173423    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75209be8_a14e_40e9_9db9_d0a276c958f2.slice/crio-9cb8890aa5ccf5a1659a2ab95831061fc789af03ca5fa66093e3a178ea57b487 WatchSource:0}: Error finding container 9cb8890aa5ccf5a1659a2ab95831061fc789af03ca5fa66093e3a178ea57b487: Status 404 returned error can't find the container with id 9cb8890aa5ccf5a1659a2ab95831061fc789af03ca5fa66093e3a178ea57b487
Oct 30 18:40:15.176482 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:15.176095    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:40:15.176482 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:15.176170    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:40:15.925022 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:15.924977    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw" event={"ID":"75209be8-a14e-40e9-9db9-d0a276c958f2","Type":"ContainerStarted","Data":"adcf09577979c12ef915b573987e39c308bf77fc668769260794824b9bc68f34"}
Oct 30 18:40:15.925469 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:15.925045    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw" event={"ID":"75209be8-a14e-40e9-9db9-d0a276c958f2","Type":"ContainerStarted","Data":"9cb8890aa5ccf5a1659a2ab95831061fc789af03ca5fa66093e3a178ea57b487"}
Oct 30 18:40:15.925469 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:15.925092    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:16.164432 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:16.164367    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw" podStartSLOduration=2.164348526 podStartE2EDuration="2.164348526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:16.114020587 +0000 UTC m=+336.825769286" watchObservedRunningTime="2026-10-30 18:40:16.164348526 +0000 UTC m=+336.876097220"
Oct 30 18:40:16.932684 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:16.932648    5820 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-94chw container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 18:40:16.933363 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:16.932703    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw" podUID="75209be8-a14e-40e9-9db9-d0a276c958f2" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:40:19.514849 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:19.514819    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:19.696290 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:19.696226    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:19.884838 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:19.884802    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:19.931606 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:19.931581    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-94chw"
Oct 30 18:40:20.060623 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.060584    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:20.155780 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.155698    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"]
Oct 30 18:40:20.155935 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.155886    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p" podUID="f8ec0830-869f-43d4-a461-e56b64b7a4ea" containerName="discovery" containerID="cri-o://f3ca58e89e93667fee1b54ddb1e8fa770637c1bcdcf80391d35a72add205eda4" gracePeriod=30
Oct 30 18:40:20.244133 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.244105    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:20.420767 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.420704    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:40:20.430305 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.430274    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:20.541210 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.541171    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-csr-ca-configmap\") pod \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") "
Oct 30 18:40:20.541716 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.541239    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8ec0830-869f-43d4-a461-e56b64b7a4ea-kube-api-access-l64xr\" (UniqueName: \"kubernetes.io/projected/f8ec0830-869f-43d4-a461-e56b64b7a4ea-kube-api-access-l64xr\") pod \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") "
Oct 30 18:40:20.541716 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.541311    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-csr-dns-cert\") pod \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") "
Oct 30 18:40:20.541716 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.541347    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8ec0830-869f-43d4-a461-e56b64b7a4ea-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8ec0830-869f-43d4-a461-e56b64b7a4ea-local-certs\") pod \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") "
Oct 30 18:40:20.541716 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.541395    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f8ec0830-869f-43d4-a461-e56b64b7a4ea-cacerts\" (UniqueName: \"kubernetes.io/secret/f8ec0830-869f-43d4-a461-e56b64b7a4ea-cacerts\") pod \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") "
Oct 30 18:40:20.541716 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.541456    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-token\" (UniqueName: \"kubernetes.io/projected/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-token\") pod \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") "
Oct 30 18:40:20.541716 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.541513    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-kubeconfig\") pod \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\" (UID: \"f8ec0830-869f-43d4-a461-e56b64b7a4ea\") "
Oct 30 18:40:20.542467 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.542392    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-csr-ca-configmap" pod "f8ec0830-869f-43d4-a461-e56b64b7a4ea" (UID: "f8ec0830-869f-43d4-a461-e56b64b7a4ea"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:40:20.544546 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.544492    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-kubeconfig" pod "f8ec0830-869f-43d4-a461-e56b64b7a4ea" (UID: "f8ec0830-869f-43d4-a461-e56b64b7a4ea"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:40:20.544685 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.544667    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-csr-dns-cert" pod "f8ec0830-869f-43d4-a461-e56b64b7a4ea" (UID: "f8ec0830-869f-43d4-a461-e56b64b7a4ea"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:40:20.545131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.545103    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8ec0830-869f-43d4-a461-e56b64b7a4ea-local-certs" pod "f8ec0830-869f-43d4-a461-e56b64b7a4ea" (UID: "f8ec0830-869f-43d4-a461-e56b64b7a4ea"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:40:20.545272 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.545248    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8ec0830-869f-43d4-a461-e56b64b7a4ea-cacerts" pod "f8ec0830-869f-43d4-a461-e56b64b7a4ea" (UID: "f8ec0830-869f-43d4-a461-e56b64b7a4ea"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:40:20.546325 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.546301    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8ec0830-869f-43d4-a461-e56b64b7a4ea-kube-api-access-l64xr" pod "f8ec0830-869f-43d4-a461-e56b64b7a4ea" (UID: "f8ec0830-869f-43d4-a461-e56b64b7a4ea"). InnerVolumeSpecName "kube-api-access-l64xr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:40:20.546325 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.546314    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-token" pod "f8ec0830-869f-43d4-a461-e56b64b7a4ea" (UID: "f8ec0830-869f-43d4-a461-e56b64b7a4ea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:40:20.643111 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.643074    5820 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-csr-dns-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:40:20.643111 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.643111    5820 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8ec0830-869f-43d4-a461-e56b64b7a4ea-local-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:40:20.643335 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.643128    5820 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f8ec0830-869f-43d4-a461-e56b64b7a4ea-cacerts\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:40:20.643335 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.643143    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:40:20.643335 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.643157    5820 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-kubeconfig\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:40:20.643335 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.643172    5820 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f8ec0830-869f-43d4-a461-e56b64b7a4ea-istio-csr-ca-configmap\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:40:20.643335 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.643186    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l64xr\" (UniqueName: \"kubernetes.io/projected/f8ec0830-869f-43d4-a461-e56b64b7a4ea-kube-api-access-l64xr\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:40:20.651516 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.651484    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:20.676701 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.676623    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-vbdd5"]
Oct 30 18:40:20.677158 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.677131    5820 cpu_manager.go:414] "containerMap: removing container" podUID="f8ec0830-869f-43d4-a461-e56b64b7a4ea" containerName="discovery"
Oct 30 18:40:20.677158 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.677155    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8ec0830-869f-43d4-a461-e56b64b7a4ea" containerName="discovery"
Oct 30 18:40:20.677333 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.677248    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8ec0830-869f-43d4-a461-e56b64b7a4ea" containerName="discovery"
Oct 30 18:40:20.680665 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.680645    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-vbdd5"
Oct 30 18:40:20.739773 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.739742    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Oct 30 18:40:20.739995 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.739836    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-nx6tb\""
Oct 30 18:40:20.741581 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.741554    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-vbdd5"]
Oct 30 18:40:20.744133 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.744109    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwztb\" (UniqueName: \"kubernetes.io/projected/2452b1f3-519b-4e1d-ad93-16c233d34895-kube-api-access-lwztb\") pod \"model-serving-api-75bd89fdfc-vbdd5\" (UID: \"2452b1f3-519b-4e1d-ad93-16c233d34895\") " pod="opendatahub/model-serving-api-75bd89fdfc-vbdd5"
Oct 30 18:40:20.744240 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.744162    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2452b1f3-519b-4e1d-ad93-16c233d34895-tls-certs\") pod \"model-serving-api-75bd89fdfc-vbdd5\" (UID: \"2452b1f3-519b-4e1d-ad93-16c233d34895\") " pod="opendatahub/model-serving-api-75bd89fdfc-vbdd5"
Oct 30 18:40:20.775511 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.775479    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-769875db56-jbdx9"]
Oct 30 18:40:20.779162 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.779144    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-jbdx9"
Oct 30 18:40:20.836260 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.836228    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-jbdx9"]
Oct 30 18:40:20.839799 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.839765    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-qdqrd\""
Oct 30 18:40:20.839965 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.839819    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Oct 30 18:40:20.839965 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.839887    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Oct 30 18:40:20.845258 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.845230    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2452b1f3-519b-4e1d-ad93-16c233d34895-tls-certs\") pod \"model-serving-api-75bd89fdfc-vbdd5\" (UID: \"2452b1f3-519b-4e1d-ad93-16c233d34895\") " pod="opendatahub/model-serving-api-75bd89fdfc-vbdd5"
Oct 30 18:40:20.845372 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.845308    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48168bb8-0b54-4f25-9139-0ddc3f7950d6-metrics-certs\") pod \"odh-model-controller-769875db56-jbdx9\" (UID: \"48168bb8-0b54-4f25-9139-0ddc3f7950d6\") " pod="opendatahub/odh-model-controller-769875db56-jbdx9"
Oct 30 18:40:20.845372 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.845343    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/48168bb8-0b54-4f25-9139-0ddc3f7950d6-cert\") pod \"odh-model-controller-769875db56-jbdx9\" (UID: \"48168bb8-0b54-4f25-9139-0ddc3f7950d6\") " pod="opendatahub/odh-model-controller-769875db56-jbdx9"
Oct 30 18:40:20.845497 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.845373    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkqvt\" (UniqueName: \"kubernetes.io/projected/48168bb8-0b54-4f25-9139-0ddc3f7950d6-kube-api-access-xkqvt\") pod \"odh-model-controller-769875db56-jbdx9\" (UID: \"48168bb8-0b54-4f25-9139-0ddc3f7950d6\") " pod="opendatahub/odh-model-controller-769875db56-jbdx9"
Oct 30 18:40:20.845497 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.845414    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwztb\" (UniqueName: \"kubernetes.io/projected/2452b1f3-519b-4e1d-ad93-16c233d34895-kube-api-access-lwztb\") pod \"model-serving-api-75bd89fdfc-vbdd5\" (UID: \"2452b1f3-519b-4e1d-ad93-16c233d34895\") " pod="opendatahub/model-serving-api-75bd89fdfc-vbdd5"
Oct 30 18:40:20.848427 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.848397    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2452b1f3-519b-4e1d-ad93-16c233d34895-tls-certs\") pod \"model-serving-api-75bd89fdfc-vbdd5\" (UID: \"2452b1f3-519b-4e1d-ad93-16c233d34895\") " pod="opendatahub/model-serving-api-75bd89fdfc-vbdd5"
Oct 30 18:40:20.906712 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.906682    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwztb\" (UniqueName: \"kubernetes.io/projected/2452b1f3-519b-4e1d-ad93-16c233d34895-kube-api-access-lwztb\") pod \"model-serving-api-75bd89fdfc-vbdd5\" (UID: \"2452b1f3-519b-4e1d-ad93-16c233d34895\") " pod="opendatahub/model-serving-api-75bd89fdfc-vbdd5"
Oct 30 18:40:20.946786 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.946712    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48168bb8-0b54-4f25-9139-0ddc3f7950d6-metrics-certs\") pod \"odh-model-controller-769875db56-jbdx9\" (UID: \"48168bb8-0b54-4f25-9139-0ddc3f7950d6\") " pod="opendatahub/odh-model-controller-769875db56-jbdx9"
Oct 30 18:40:20.946786 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.946754    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/48168bb8-0b54-4f25-9139-0ddc3f7950d6-cert\") pod \"odh-model-controller-769875db56-jbdx9\" (UID: \"48168bb8-0b54-4f25-9139-0ddc3f7950d6\") " pod="opendatahub/odh-model-controller-769875db56-jbdx9"
Oct 30 18:40:20.947041 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.946922    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkqvt\" (UniqueName: \"kubernetes.io/projected/48168bb8-0b54-4f25-9139-0ddc3f7950d6-kube-api-access-xkqvt\") pod \"odh-model-controller-769875db56-jbdx9\" (UID: \"48168bb8-0b54-4f25-9139-0ddc3f7950d6\") " pod="opendatahub/odh-model-controller-769875db56-jbdx9"
Oct 30 18:40:20.949790 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.949760    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/48168bb8-0b54-4f25-9139-0ddc3f7950d6-cert\") pod \"odh-model-controller-769875db56-jbdx9\" (UID: \"48168bb8-0b54-4f25-9139-0ddc3f7950d6\") " pod="opendatahub/odh-model-controller-769875db56-jbdx9"
Oct 30 18:40:20.949790 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.949786    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48168bb8-0b54-4f25-9139-0ddc3f7950d6-metrics-certs\") pod \"odh-model-controller-769875db56-jbdx9\" (UID: \"48168bb8-0b54-4f25-9139-0ddc3f7950d6\") " pod="opendatahub/odh-model-controller-769875db56-jbdx9"
Oct 30 18:40:20.953890 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.953866    5820 generic.go:356] "Generic (PLEG): container finished" podID="f8ec0830-869f-43d4-a461-e56b64b7a4ea" containerID="f3ca58e89e93667fee1b54ddb1e8fa770637c1bcdcf80391d35a72add205eda4" exitCode=0
Oct 30 18:40:20.954026 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.953915    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"
Oct 30 18:40:20.954026 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.953982    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p" event={"ID":"f8ec0830-869f-43d4-a461-e56b64b7a4ea","Type":"ContainerDied","Data":"f3ca58e89e93667fee1b54ddb1e8fa770637c1bcdcf80391d35a72add205eda4"}
Oct 30 18:40:20.954120 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.954042    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p" event={"ID":"f8ec0830-869f-43d4-a461-e56b64b7a4ea","Type":"ContainerDied","Data":"f8d9d8dcefa33b16114a2e6ec880c169c86957a816c443996736d153a9a76ce7"}
Oct 30 18:40:20.954120 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.954076    5820 scope.go:122] "RemoveContainer" containerID="f3ca58e89e93667fee1b54ddb1e8fa770637c1bcdcf80391d35a72add205eda4"
Oct 30 18:40:20.964346 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.964324    5820 scope.go:122] "RemoveContainer" containerID="f3ca58e89e93667fee1b54ddb1e8fa770637c1bcdcf80391d35a72add205eda4"
Oct 30 18:40:20.964626 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:40:20.964601    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3ca58e89e93667fee1b54ddb1e8fa770637c1bcdcf80391d35a72add205eda4\": container with ID starting with f3ca58e89e93667fee1b54ddb1e8fa770637c1bcdcf80391d35a72add205eda4 not found: ID does not exist" containerID="f3ca58e89e93667fee1b54ddb1e8fa770637c1bcdcf80391d35a72add205eda4"
Oct 30 18:40:20.964741 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.964638    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3ca58e89e93667fee1b54ddb1e8fa770637c1bcdcf80391d35a72add205eda4"} err="failed to get container status \"f3ca58e89e93667fee1b54ddb1e8fa770637c1bcdcf80391d35a72add205eda4\": rpc error: code = NotFound desc = could not find container \"f3ca58e89e93667fee1b54ddb1e8fa770637c1bcdcf80391d35a72add205eda4\": container with ID starting with f3ca58e89e93667fee1b54ddb1e8fa770637c1bcdcf80391d35a72add205eda4 not found: ID does not exist"
Oct 30 18:40:20.995510 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:20.995479    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-vbdd5"
Oct 30 18:40:21.009213 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:21.009189    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkqvt\" (UniqueName: \"kubernetes.io/projected/48168bb8-0b54-4f25-9139-0ddc3f7950d6-kube-api-access-xkqvt\") pod \"odh-model-controller-769875db56-jbdx9\" (UID: \"48168bb8-0b54-4f25-9139-0ddc3f7950d6\") " pod="opendatahub/odh-model-controller-769875db56-jbdx9"
Oct 30 18:40:21.091922 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:21.091884    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-jbdx9"
Oct 30 18:40:21.126285 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:21.126248    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"]
Oct 30 18:40:21.131351 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:21.131324    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-lz85p"]
Oct 30 18:40:21.380303 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:21.380264    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-vbdd5"]
Oct 30 18:40:21.383911 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:40:21.383882    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2452b1f3_519b_4e1d_ad93_16c233d34895.slice/crio-0cd74e428341ee23fae59fc283a8b441f06aa1cef34dca381f86175a715836c3 WatchSource:0}: Error finding container 0cd74e428341ee23fae59fc283a8b441f06aa1cef34dca381f86175a715836c3: Status 404 returned error can't find the container with id 0cd74e428341ee23fae59fc283a8b441f06aa1cef34dca381f86175a715836c3
Oct 30 18:40:21.438424 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:21.438383    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f8ec0830-869f-43d4-a461-e56b64b7a4ea" path="/var/lib/kubelet/pods/f8ec0830-869f-43d4-a461-e56b64b7a4ea/volumes"
Oct 30 18:40:21.476683 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:21.476652    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-jbdx9"]
Oct 30 18:40:21.480683 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:40:21.480656    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48168bb8_0b54_4f25_9139_0ddc3f7950d6.slice/crio-6d4b1d621d90485c8de5a4331afc554215563be36626a4687c3af21ecb085747 WatchSource:0}: Error finding container 6d4b1d621d90485c8de5a4331afc554215563be36626a4687c3af21ecb085747: Status 404 returned error can't find the container with id 6d4b1d621d90485c8de5a4331afc554215563be36626a4687c3af21ecb085747
Oct 30 18:40:21.856336 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:21.856288    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:21.961623 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:21.961585    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-jbdx9" event={"ID":"48168bb8-0b54-4f25-9139-0ddc3f7950d6","Type":"ContainerStarted","Data":"6d4b1d621d90485c8de5a4331afc554215563be36626a4687c3af21ecb085747"}
Oct 30 18:40:21.962976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:21.962928    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-vbdd5" event={"ID":"2452b1f3-519b-4e1d-ad93-16c233d34895","Type":"ContainerStarted","Data":"0cd74e428341ee23fae59fc283a8b441f06aa1cef34dca381f86175a715836c3"}
Oct 30 18:40:24.985274 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:24.985233    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-vbdd5" event={"ID":"2452b1f3-519b-4e1d-ad93-16c233d34895","Type":"ContainerStarted","Data":"fc14868291310f02b5b0b4fd96f39318858d98e1779332231a4a5ba7945fe590"}
Oct 30 18:40:24.986140 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:24.986118    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-vbdd5"
Oct 30 18:40:25.104004 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:25.103924    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-vbdd5" podStartSLOduration=2.200118944 podStartE2EDuration="5.103904024s" totalImagesPullingTime="2.90378508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:25.102034137 +0000 UTC m=+345.813782834" watchObservedRunningTime="2026-10-30 18:40:25.103904024 +0000 UTC m=+345.815652721"
Oct 30 18:40:25.992677 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:25.992637    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-jbdx9" event={"ID":"48168bb8-0b54-4f25-9139-0ddc3f7950d6","Type":"ContainerStarted","Data":"14f8e3fabd0cbea9dd80e6e55f9aa76bf9b9ba2bdcc8f0e7e0added32fa92960"}
Oct 30 18:40:26.111061 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:26.111005    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-769875db56-jbdx9" podStartSLOduration=2.382389933 podStartE2EDuration="6.110984594s" totalImagesPullingTime="3.728594661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:26.11041189 +0000 UTC m=+346.822160587" watchObservedRunningTime="2026-10-30 18:40:26.110984594 +0000 UTC m=+346.822733292"
Oct 30 18:40:26.998442 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:26.998407    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw" event={"ID":"d0e5ae11-c274-4e34-a6fe-ac8df2700247","Type":"ContainerStarted","Data":"8fe6b0b2636251b05708257f350fb16ec5acbcd71f3fc26f3cda80b4eb26ce3d"}
Oct 30 18:40:26.998848 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:26.998506    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-jbdx9"
Oct 30 18:40:27.124037 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:27.123920    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw" podStartSLOduration=2.709724542 podStartE2EDuration="19.123901898s" totalImagesPullingTime="16.414177356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:27.123371027 +0000 UTC m=+347.835119741" watchObservedRunningTime="2026-10-30 18:40:27.123901898 +0000 UTC m=+347.835650596"
Oct 30 18:40:27.293277 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:27.293241    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:27.521431 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:27.521342    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:27.716716 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:27.716678    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:27.893712 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:27.893664    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:28.002809 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:28.002776    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw"
Oct 30 18:40:28.066283 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:28.066248    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:28.316516 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:28.316481    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:28.492590 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:28.492562    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:28.662156 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:28.662066    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:36.958349 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:36.958312    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:37.006985 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:37.006937    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-vbdd5"
Oct 30 18:40:37.141553 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:37.141520    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:38.005297 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:38.005270    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-769875db56-jbdx9"
Oct 30 18:40:38.207325 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:40:38.207289    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:40:39.005689 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:39.005650    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:39.016384 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:39.016354    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mcgqw"
Oct 30 18:40:39.227663 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:39.227629    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:39.433637 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:39.433600    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:40.071335 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:40.071300    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"]
Oct 30 18:40:40.076092 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:40.076071    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"
Oct 30 18:40:40.128562 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:40.128537    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 18:40:40.128562 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:40.128557    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 18:40:40.128744 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:40.128576    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gxfk7\""
Oct 30 18:40:40.134211 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:40.134183    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"]
Oct 30 18:40:40.223465 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:40.223438    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh9f7\" (UniqueName: \"kubernetes.io/projected/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-kube-api-access-xh9f7\") pod \"maas-api-78ffbd6b58-f5hbm\" (UID: \"6129a7e4-9499-4c74-b456-a9bd34a2f2f5\") " pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"
Oct 30 18:40:40.223628 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:40.223486    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-openshift-service-ca\") pod \"maas-api-78ffbd6b58-f5hbm\" (UID: \"6129a7e4-9499-4c74-b456-a9bd34a2f2f5\") " pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"
Oct 30 18:40:40.223628 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:40.223513    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-maas-api-tls\") pod \"maas-api-78ffbd6b58-f5hbm\" (UID: \"6129a7e4-9499-4c74-b456-a9bd34a2f2f5\") " pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"
Oct 30 18:40:40.223628 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:40.223600    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-metrics-certs\") pod \"maas-api-78ffbd6b58-f5hbm\" (UID: \"6129a7e4-9499-4c74-b456-a9bd34a2f2f5\") " pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"
Oct 30 18:40:40.324628 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:40.324543    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-metrics-certs\") pod \"maas-api-78ffbd6b58-f5hbm\" (UID: \"6129a7e4-9499-4c74-b456-a9bd34a2f2f5\") " pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"
Oct 30 18:40:40.324628 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:40.324618    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh9f7\" (UniqueName: \"kubernetes.io/projected/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-kube-api-access-xh9f7\") pod \"maas-api-78ffbd6b58-f5hbm\" (UID: \"6129a7e4-9499-4c74-b456-a9bd34a2f2f5\") " pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"
Oct 30 18:40:40.324850 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:40.324675    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-openshift-service-ca\") pod \"maas-api-78ffbd6b58-f5hbm\" (UID: \"6129a7e4-9499-4c74-b456-a9bd34a2f2f5\") " pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"
Oct 30 18:40:40.324850 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:40.324710    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-maas-api-tls\") pod \"maas-api-78ffbd6b58-f5hbm\" (UID: \"6129a7e4-9499-4c74-b456-a9bd34a2f2f5\") " pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"
Oct 30 18:40:40.325388 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:40.325361    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-openshift-service-ca\") pod \"maas-api-78ffbd6b58-f5hbm\" (UID: \"6129a7e4-9499-4c74-b456-a9bd34a2f2f5\") " pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"
Oct 30 18:40:40.327231 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:40.327207    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-maas-api-tls\") pod \"maas-api-78ffbd6b58-f5hbm\" (UID: \"6129a7e4-9499-4c74-b456-a9bd34a2f2f5\") " pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"
Oct 30 18:40:40.327323 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:40.327226    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-metrics-certs\") pod \"maas-api-78ffbd6b58-f5hbm\" (UID: \"6129a7e4-9499-4c74-b456-a9bd34a2f2f5\") " pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"
Oct 30 18:40:40.385860 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:40.385825    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh9f7\" (UniqueName: \"kubernetes.io/projected/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-kube-api-access-xh9f7\") pod \"maas-api-78ffbd6b58-f5hbm\" (UID: \"6129a7e4-9499-4c74-b456-a9bd34a2f2f5\") " pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"
Oct 30 18:40:40.388732 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:40.388711    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"
Oct 30 18:40:40.764208 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:40.763989    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"]
Oct 30 18:40:40.766499 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:40:40.766474    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6129a7e4_9499_4c74_b456_a9bd34a2f2f5.slice/crio-c392557ecdfe9998545ceffcabd2568f4cb6d712113933474a2fc9fe71d8aecc WatchSource:0}: Error finding container c392557ecdfe9998545ceffcabd2568f4cb6d712113933474a2fc9fe71d8aecc: Status 404 returned error can't find the container with id c392557ecdfe9998545ceffcabd2568f4cb6d712113933474a2fc9fe71d8aecc
Oct 30 18:40:41.061069 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:41.061038    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm" event={"ID":"6129a7e4-9499-4c74-b456-a9bd34a2f2f5","Type":"ContainerStarted","Data":"c392557ecdfe9998545ceffcabd2568f4cb6d712113933474a2fc9fe71d8aecc"}
Oct 30 18:40:47.089277 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:47.089244    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm" event={"ID":"6129a7e4-9499-4c74-b456-a9bd34a2f2f5","Type":"ContainerStarted","Data":"4aa1c376dc36fcadda2c01105302fbd2dfdbd3824c31a0e597226d02242aab30"}
Oct 30 18:40:47.089740 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:47.089383    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"
Oct 30 18:40:47.260596 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:47.260475    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm" podStartSLOduration=1.064219804 podStartE2EDuration="7.260463153s" totalImagesPullingTime="6.196243349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:47.254939619 +0000 UTC m=+367.966688316" watchObservedRunningTime="2026-10-30 18:40:47.260463153 +0000 UTC m=+367.972211849"
Oct 30 18:40:48.202188 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:48.202153    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:48.440672 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:48.440637    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:53.100098 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:53.100071    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"
Oct 30 18:40:53.744256 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:53.744204    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:53.976310 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:53.976266    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:54.307280 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:54.307245    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-776f8954dc-s9lws"]
Oct 30 18:40:54.405910 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:54.405874    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-776f8954dc-s9lws"]
Oct 30 18:40:54.406071 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:54.406021    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-776f8954dc-s9lws"
Oct 30 18:40:54.453127 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:54.453091    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60db3aee-1f1d-404f-abfb-e6f2ea221b3e-metrics-certs\") pod \"maas-api-776f8954dc-s9lws\" (UID: \"60db3aee-1f1d-404f-abfb-e6f2ea221b3e\") " pod="odh-ai-gateway-infra/maas-api-776f8954dc-s9lws"
Oct 30 18:40:54.453265 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:54.453147    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jznq\" (UniqueName: \"kubernetes.io/projected/60db3aee-1f1d-404f-abfb-e6f2ea221b3e-kube-api-access-5jznq\") pod \"maas-api-776f8954dc-s9lws\" (UID: \"60db3aee-1f1d-404f-abfb-e6f2ea221b3e\") " pod="odh-ai-gateway-infra/maas-api-776f8954dc-s9lws"
Oct 30 18:40:54.453304 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:54.453286    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60db3aee-1f1d-404f-abfb-e6f2ea221b3e-maas-api-tls\") pod \"maas-api-776f8954dc-s9lws\" (UID: \"60db3aee-1f1d-404f-abfb-e6f2ea221b3e\") " pod="odh-ai-gateway-infra/maas-api-776f8954dc-s9lws"
Oct 30 18:40:54.453339 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:54.453312    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60db3aee-1f1d-404f-abfb-e6f2ea221b3e-openshift-service-ca\") pod \"maas-api-776f8954dc-s9lws\" (UID: \"60db3aee-1f1d-404f-abfb-e6f2ea221b3e\") " pod="odh-ai-gateway-infra/maas-api-776f8954dc-s9lws"
Oct 30 18:40:54.553814 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:54.553785    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60db3aee-1f1d-404f-abfb-e6f2ea221b3e-maas-api-tls\") pod \"maas-api-776f8954dc-s9lws\" (UID: \"60db3aee-1f1d-404f-abfb-e6f2ea221b3e\") " pod="odh-ai-gateway-infra/maas-api-776f8954dc-s9lws"
Oct 30 18:40:54.554095 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:54.553823    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60db3aee-1f1d-404f-abfb-e6f2ea221b3e-openshift-service-ca\") pod \"maas-api-776f8954dc-s9lws\" (UID: \"60db3aee-1f1d-404f-abfb-e6f2ea221b3e\") " pod="odh-ai-gateway-infra/maas-api-776f8954dc-s9lws"
Oct 30 18:40:54.554095 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:54.553846    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60db3aee-1f1d-404f-abfb-e6f2ea221b3e-metrics-certs\") pod \"maas-api-776f8954dc-s9lws\" (UID: \"60db3aee-1f1d-404f-abfb-e6f2ea221b3e\") " pod="odh-ai-gateway-infra/maas-api-776f8954dc-s9lws"
Oct 30 18:40:54.554095 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:54.553871    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jznq\" (UniqueName: \"kubernetes.io/projected/60db3aee-1f1d-404f-abfb-e6f2ea221b3e-kube-api-access-5jznq\") pod \"maas-api-776f8954dc-s9lws\" (UID: \"60db3aee-1f1d-404f-abfb-e6f2ea221b3e\") " pod="odh-ai-gateway-infra/maas-api-776f8954dc-s9lws"
Oct 30 18:40:54.554450 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:54.554431    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60db3aee-1f1d-404f-abfb-e6f2ea221b3e-openshift-service-ca\") pod \"maas-api-776f8954dc-s9lws\" (UID: \"60db3aee-1f1d-404f-abfb-e6f2ea221b3e\") " pod="odh-ai-gateway-infra/maas-api-776f8954dc-s9lws"
Oct 30 18:40:54.556531 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:54.556508    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60db3aee-1f1d-404f-abfb-e6f2ea221b3e-metrics-certs\") pod \"maas-api-776f8954dc-s9lws\" (UID: \"60db3aee-1f1d-404f-abfb-e6f2ea221b3e\") " pod="odh-ai-gateway-infra/maas-api-776f8954dc-s9lws"
Oct 30 18:40:54.556622 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:54.556553    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60db3aee-1f1d-404f-abfb-e6f2ea221b3e-maas-api-tls\") pod \"maas-api-776f8954dc-s9lws\" (UID: \"60db3aee-1f1d-404f-abfb-e6f2ea221b3e\") " pod="odh-ai-gateway-infra/maas-api-776f8954dc-s9lws"
Oct 30 18:40:54.615240 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:54.615180    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jznq\" (UniqueName: \"kubernetes.io/projected/60db3aee-1f1d-404f-abfb-e6f2ea221b3e-kube-api-access-5jznq\") pod \"maas-api-776f8954dc-s9lws\" (UID: \"60db3aee-1f1d-404f-abfb-e6f2ea221b3e\") " pod="odh-ai-gateway-infra/maas-api-776f8954dc-s9lws"
Oct 30 18:40:54.754115 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:54.754083    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-776f8954dc-s9lws"
Oct 30 18:40:55.167063 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:55.167033    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-776f8954dc-s9lws"]
Oct 30 18:40:55.170427 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:40:55.170399    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60db3aee_1f1d_404f_abfb_e6f2ea221b3e.slice/crio-371c71ed90e43c08f7b2d5acc62b88ec23a16f928eff59f8040c761aadb12b2f WatchSource:0}: Error finding container 371c71ed90e43c08f7b2d5acc62b88ec23a16f928eff59f8040c761aadb12b2f: Status 404 returned error can't find the container with id 371c71ed90e43c08f7b2d5acc62b88ec23a16f928eff59f8040c761aadb12b2f
Oct 30 18:40:56.130699 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:56.130665    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-776f8954dc-s9lws" event={"ID":"60db3aee-1f1d-404f-abfb-e6f2ea221b3e","Type":"ContainerStarted","Data":"371c71ed90e43c08f7b2d5acc62b88ec23a16f928eff59f8040c761aadb12b2f"}
Oct 30 18:40:57.136443 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:57.136408    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-776f8954dc-s9lws" event={"ID":"60db3aee-1f1d-404f-abfb-e6f2ea221b3e","Type":"ContainerStarted","Data":"8caed99d62ad6145c711b561f1a889ce1aade74de084a522ec47852b8ebd364a"}
Oct 30 18:40:57.136896 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:57.136475    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-776f8954dc-s9lws"
Oct 30 18:40:57.260677 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:57.260626    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-776f8954dc-s9lws" podStartSLOduration=2.090411148 podStartE2EDuration="3.260612264s" totalImagesPullingTime="1.170201116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:57.259229044 +0000 UTC m=+377.970977745" watchObservedRunningTime="2026-10-30 18:40:57.260612264 +0000 UTC m=+377.972360976"
Oct 30 18:40:58.194400 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:58.194369    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-5w6kz"]
Oct 30 18:40:58.194897 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:58.194542    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-5w6kz" podUID="9eaa27bd-09dc-46f4-acee-652fc47950fb" containerName="authorino" containerID="cri-o://7ac1d826aba8ab8dfc2fd8f12cebe0d8c2c33e72447cb93c738c88d9544eba40" gracePeriod=30
Oct 30 18:40:58.453781 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:58.453718    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-5w6kz"
Oct 30 18:40:58.490917 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:58.490888    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9eaa27bd-09dc-46f4-acee-652fc47950fb-kube-api-access-ksglr\" (UniqueName: \"kubernetes.io/projected/9eaa27bd-09dc-46f4-acee-652fc47950fb-kube-api-access-ksglr\") pod \"9eaa27bd-09dc-46f4-acee-652fc47950fb\" (UID: \"9eaa27bd-09dc-46f4-acee-652fc47950fb\") "
Oct 30 18:40:58.493045 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:58.493022    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9eaa27bd-09dc-46f4-acee-652fc47950fb-kube-api-access-ksglr" pod "9eaa27bd-09dc-46f4-acee-652fc47950fb" (UID: "9eaa27bd-09dc-46f4-acee-652fc47950fb"). InnerVolumeSpecName "kube-api-access-ksglr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:40:58.592420 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:58.592382    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksglr\" (UniqueName: \"kubernetes.io/projected/9eaa27bd-09dc-46f4-acee-652fc47950fb-kube-api-access-ksglr\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:40:59.146610 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:59.146578    5820 generic.go:356] "Generic (PLEG): container finished" podID="9eaa27bd-09dc-46f4-acee-652fc47950fb" containerID="7ac1d826aba8ab8dfc2fd8f12cebe0d8c2c33e72447cb93c738c88d9544eba40" exitCode=0
Oct 30 18:40:59.146801 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:59.146627    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-5w6kz"
Oct 30 18:40:59.146801 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:59.146663    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-5w6kz" event={"ID":"9eaa27bd-09dc-46f4-acee-652fc47950fb","Type":"ContainerDied","Data":"7ac1d826aba8ab8dfc2fd8f12cebe0d8c2c33e72447cb93c738c88d9544eba40"}
Oct 30 18:40:59.146801 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:59.146707    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-5w6kz" event={"ID":"9eaa27bd-09dc-46f4-acee-652fc47950fb","Type":"ContainerDied","Data":"8dae66552ca251e13b4a7b64288408443df0901805d5f6e0249afe178126842d"}
Oct 30 18:40:59.146801 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:59.146724    5820 scope.go:122] "RemoveContainer" containerID="7ac1d826aba8ab8dfc2fd8f12cebe0d8c2c33e72447cb93c738c88d9544eba40"
Oct 30 18:40:59.155888 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:59.155874    5820 scope.go:122] "RemoveContainer" containerID="7ac1d826aba8ab8dfc2fd8f12cebe0d8c2c33e72447cb93c738c88d9544eba40"
Oct 30 18:40:59.156185 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:40:59.156165    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ac1d826aba8ab8dfc2fd8f12cebe0d8c2c33e72447cb93c738c88d9544eba40\": container with ID starting with 7ac1d826aba8ab8dfc2fd8f12cebe0d8c2c33e72447cb93c738c88d9544eba40 not found: ID does not exist" containerID="7ac1d826aba8ab8dfc2fd8f12cebe0d8c2c33e72447cb93c738c88d9544eba40"
Oct 30 18:40:59.156250 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:59.156190    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ac1d826aba8ab8dfc2fd8f12cebe0d8c2c33e72447cb93c738c88d9544eba40"} err="failed to get container status \"7ac1d826aba8ab8dfc2fd8f12cebe0d8c2c33e72447cb93c738c88d9544eba40\": rpc error: code = NotFound desc = could not find container \"7ac1d826aba8ab8dfc2fd8f12cebe0d8c2c33e72447cb93c738c88d9544eba40\": container with ID starting with 7ac1d826aba8ab8dfc2fd8f12cebe0d8c2c33e72447cb93c738c88d9544eba40 not found: ID does not exist"
Oct 30 18:40:59.312489 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:59.312449    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-5w6kz"]
Oct 30 18:40:59.361751 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:59.361719    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-5w6kz"]
Oct 30 18:40:59.434244 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:40:59.434167    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9eaa27bd-09dc-46f4-acee-652fc47950fb" path="/var/lib/kubelet/pods/9eaa27bd-09dc-46f4-acee-652fc47950fb/volumes"
Oct 30 18:41:03.145774 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:03.145746    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-776f8954dc-s9lws"
Oct 30 18:41:03.314858 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:03.314809    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"]
Oct 30 18:41:03.315187 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:03.315132    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm" podUID="6129a7e4-9499-4c74-b456-a9bd34a2f2f5" containerName="maas-api" containerID="cri-o://4aa1c376dc36fcadda2c01105302fbd2dfdbd3824c31a0e597226d02242aab30" gracePeriod=30
Oct 30 18:41:03.562899 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:03.562865    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"
Oct 30 18:41:03.632976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:03.632921    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-kube-api-access-xh9f7\" (UniqueName: \"kubernetes.io/projected/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-kube-api-access-xh9f7\") pod \"6129a7e4-9499-4c74-b456-a9bd34a2f2f5\" (UID: \"6129a7e4-9499-4c74-b456-a9bd34a2f2f5\") "
Oct 30 18:41:03.633140 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:03.633031    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-openshift-service-ca\") pod \"6129a7e4-9499-4c74-b456-a9bd34a2f2f5\" (UID: \"6129a7e4-9499-4c74-b456-a9bd34a2f2f5\") "
Oct 30 18:41:03.633140 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:03.633068    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-maas-api-tls\") pod \"6129a7e4-9499-4c74-b456-a9bd34a2f2f5\" (UID: \"6129a7e4-9499-4c74-b456-a9bd34a2f2f5\") "
Oct 30 18:41:03.633140 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:03.633095    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-metrics-certs\") pod \"6129a7e4-9499-4c74-b456-a9bd34a2f2f5\" (UID: \"6129a7e4-9499-4c74-b456-a9bd34a2f2f5\") "
Oct 30 18:41:03.633442 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:03.633412    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-openshift-service-ca" pod "6129a7e4-9499-4c74-b456-a9bd34a2f2f5" (UID: "6129a7e4-9499-4c74-b456-a9bd34a2f2f5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:41:03.635328 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:03.635303    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-metrics-certs" pod "6129a7e4-9499-4c74-b456-a9bd34a2f2f5" (UID: "6129a7e4-9499-4c74-b456-a9bd34a2f2f5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:41:03.635433 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:03.635358    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-maas-api-tls" pod "6129a7e4-9499-4c74-b456-a9bd34a2f2f5" (UID: "6129a7e4-9499-4c74-b456-a9bd34a2f2f5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:41:03.635488 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:03.635469    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-kube-api-access-xh9f7" pod "6129a7e4-9499-4c74-b456-a9bd34a2f2f5" (UID: "6129a7e4-9499-4c74-b456-a9bd34a2f2f5"). InnerVolumeSpecName "kube-api-access-xh9f7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:41:03.734010 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:03.733917    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xh9f7\" (UniqueName: \"kubernetes.io/projected/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-kube-api-access-xh9f7\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:41:03.734010 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:03.733966    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:41:03.734010 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:03.733977    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:41:03.734010 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:03.734004    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6129a7e4-9499-4c74-b456-a9bd34a2f2f5-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:41:04.170470 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:04.170435    5820 generic.go:356] "Generic (PLEG): container finished" podID="6129a7e4-9499-4c74-b456-a9bd34a2f2f5" containerID="4aa1c376dc36fcadda2c01105302fbd2dfdbd3824c31a0e597226d02242aab30" exitCode=0
Oct 30 18:41:04.170470 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:04.170473    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm" event={"ID":"6129a7e4-9499-4c74-b456-a9bd34a2f2f5","Type":"ContainerDied","Data":"4aa1c376dc36fcadda2c01105302fbd2dfdbd3824c31a0e597226d02242aab30"}
Oct 30 18:41:04.170930 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:04.170481    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"
Oct 30 18:41:04.170930 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:04.170517    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm" event={"ID":"6129a7e4-9499-4c74-b456-a9bd34a2f2f5","Type":"ContainerDied","Data":"c392557ecdfe9998545ceffcabd2568f4cb6d712113933474a2fc9fe71d8aecc"}
Oct 30 18:41:04.170930 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:04.170542    5820 scope.go:122] "RemoveContainer" containerID="4aa1c376dc36fcadda2c01105302fbd2dfdbd3824c31a0e597226d02242aab30"
Oct 30 18:41:04.179826 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:04.179807    5820 scope.go:122] "RemoveContainer" containerID="4aa1c376dc36fcadda2c01105302fbd2dfdbd3824c31a0e597226d02242aab30"
Oct 30 18:41:04.180097 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:41:04.180077    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4aa1c376dc36fcadda2c01105302fbd2dfdbd3824c31a0e597226d02242aab30\": container with ID starting with 4aa1c376dc36fcadda2c01105302fbd2dfdbd3824c31a0e597226d02242aab30 not found: ID does not exist" containerID="4aa1c376dc36fcadda2c01105302fbd2dfdbd3824c31a0e597226d02242aab30"
Oct 30 18:41:04.180149 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:04.180108    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aa1c376dc36fcadda2c01105302fbd2dfdbd3824c31a0e597226d02242aab30"} err="failed to get container status \"4aa1c376dc36fcadda2c01105302fbd2dfdbd3824c31a0e597226d02242aab30\": rpc error: code = NotFound desc = could not find container \"4aa1c376dc36fcadda2c01105302fbd2dfdbd3824c31a0e597226d02242aab30\": container with ID starting with 4aa1c376dc36fcadda2c01105302fbd2dfdbd3824c31a0e597226d02242aab30 not found: ID does not exist"
Oct 30 18:41:04.333536 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:04.333502    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"]
Oct 30 18:41:04.382809 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:04.382779    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-78ffbd6b58-f5hbm"]
Oct 30 18:41:05.435195 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:05.435161    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6129a7e4-9499-4c74-b456-a9bd34a2f2f5" path="/var/lib/kubelet/pods/6129a7e4-9499-4c74-b456-a9bd34a2f2f5/volumes"
Oct 30 18:41:08.201619 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:41:08.201579    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:41:38.202212 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:41:38.202172    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:41:38.710522 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:38.710486    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:41:38.884997 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:38.884965    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:41:43.985412 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:43.985374    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:41:48.536807 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:41:48.536768    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:08.202390 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:42:08.202345    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:42:14.441580 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:14.441541    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-2qfm6"]
Oct 30 18:42:14.442215 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:14.442131    5820 cpu_manager.go:414] "containerMap: removing container" podUID="9eaa27bd-09dc-46f4-acee-652fc47950fb" containerName="authorino"
Oct 30 18:42:14.442215 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:14.442150    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9eaa27bd-09dc-46f4-acee-652fc47950fb" containerName="authorino"
Oct 30 18:42:14.442215 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:14.442169    5820 cpu_manager.go:414] "containerMap: removing container" podUID="6129a7e4-9499-4c74-b456-a9bd34a2f2f5" containerName="maas-api"
Oct 30 18:42:14.442215 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:14.442176    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6129a7e4-9499-4c74-b456-a9bd34a2f2f5" containerName="maas-api"
Oct 30 18:42:14.442439 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:14.442272    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="9eaa27bd-09dc-46f4-acee-652fc47950fb" containerName="authorino"
Oct 30 18:42:14.442439 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:14.442285    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="6129a7e4-9499-4c74-b456-a9bd34a2f2f5" containerName="maas-api"
Oct 30 18:42:14.445463 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:14.445442    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-2qfm6"
Oct 30 18:42:14.478547 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:14.478506    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:14.498832 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:14.498809    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Oct 30 18:42:14.522901 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:14.522866    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcjxg\" (UniqueName: \"kubernetes.io/projected/69426d3b-657a-4988-9dde-7dce4499e30d-kube-api-access-fcjxg\") pod \"ai-gateway-controller-7d85675f98-2qfm6\" (UID: \"69426d3b-657a-4988-9dde-7dce4499e30d\") " pod="opendatahub/ai-gateway-controller-7d85675f98-2qfm6"
Oct 30 18:42:14.523068 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:14.522984    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69426d3b-657a-4988-9dde-7dce4499e30d-metrics-certs\") pod \"ai-gateway-controller-7d85675f98-2qfm6\" (UID: \"69426d3b-657a-4988-9dde-7dce4499e30d\") " pod="opendatahub/ai-gateway-controller-7d85675f98-2qfm6"
Oct 30 18:42:14.548298 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:14.548265    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Oct 30 18:42:14.548298 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:14.548283    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-6xh4n\""
Oct 30 18:42:14.549566 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:14.549545    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-2qfm6"]
Oct 30 18:42:14.623693 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:14.623660    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcjxg\" (UniqueName: \"kubernetes.io/projected/69426d3b-657a-4988-9dde-7dce4499e30d-kube-api-access-fcjxg\") pod \"ai-gateway-controller-7d85675f98-2qfm6\" (UID: \"69426d3b-657a-4988-9dde-7dce4499e30d\") " pod="opendatahub/ai-gateway-controller-7d85675f98-2qfm6"
Oct 30 18:42:14.623865 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:14.623723    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69426d3b-657a-4988-9dde-7dce4499e30d-metrics-certs\") pod \"ai-gateway-controller-7d85675f98-2qfm6\" (UID: \"69426d3b-657a-4988-9dde-7dce4499e30d\") " pod="opendatahub/ai-gateway-controller-7d85675f98-2qfm6"
Oct 30 18:42:14.626231 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:14.626208    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69426d3b-657a-4988-9dde-7dce4499e30d-metrics-certs\") pod \"ai-gateway-controller-7d85675f98-2qfm6\" (UID: \"69426d3b-657a-4988-9dde-7dce4499e30d\") " pod="opendatahub/ai-gateway-controller-7d85675f98-2qfm6"
Oct 30 18:42:14.684869 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:14.684843    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcjxg\" (UniqueName: \"kubernetes.io/projected/69426d3b-657a-4988-9dde-7dce4499e30d-kube-api-access-fcjxg\") pod \"ai-gateway-controller-7d85675f98-2qfm6\" (UID: \"69426d3b-657a-4988-9dde-7dce4499e30d\") " pod="opendatahub/ai-gateway-controller-7d85675f98-2qfm6"
Oct 30 18:42:14.710324 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:14.710254    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:14.757898 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:14.757881    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-2qfm6"
Oct 30 18:42:15.128398 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:15.128371    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-2qfm6"]
Oct 30 18:42:15.130822 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:42:15.130789    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69426d3b_657a_4988_9dde_7dce4499e30d.slice/crio-774105d54e10979d1e8b91c48ed31957589ad761e2f89740b6a1812176a21a54 WatchSource:0}: Error finding container 774105d54e10979d1e8b91c48ed31957589ad761e2f89740b6a1812176a21a54: Status 404 returned error can't find the container with id 774105d54e10979d1e8b91c48ed31957589ad761e2f89740b6a1812176a21a54
Oct 30 18:42:15.472294 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:15.472202    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-2qfm6" event={"ID":"69426d3b-657a-4988-9dde-7dce4499e30d","Type":"ContainerStarted","Data":"774105d54e10979d1e8b91c48ed31957589ad761e2f89740b6a1812176a21a54"}
Oct 30 18:42:20.500824 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:20.500742    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-2qfm6" event={"ID":"69426d3b-657a-4988-9dde-7dce4499e30d","Type":"ContainerStarted","Data":"5c246b4d0bf4e56eb983b148787620a2ffbc33bdfb2e10bd9b4d33f860de2f0c"}
Oct 30 18:42:20.501211 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:20.500888    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-7d85675f98-2qfm6"
Oct 30 18:42:20.618732 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:20.618674    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-7d85675f98-2qfm6" podStartSLOduration=1.530687508 podStartE2EDuration="6.618660633s" totalImagesPullingTime="5.087973125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:20.616761771 +0000 UTC m=+461.328510669" watchObservedRunningTime="2026-10-30 18:42:20.618660633 +0000 UTC m=+461.330409330"
Oct 30 18:42:22.992829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:22.992785    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:23.165914 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.165881    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:23.367324 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.367291    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-qmbs6"]
Oct 30 18:42:23.370789 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.370771    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-qmbs6"
Oct 30 18:42:23.472016 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.471988    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Oct 30 18:42:23.472016 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.472004    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-c672p\""
Oct 30 18:42:23.473307 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.473287    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-qmbs6"]
Oct 30 18:42:23.508600 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.508568    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e933bb3-b9ba-4dff-841e-831b32f20537-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-qmbs6\" (UID: \"3e933bb3-b9ba-4dff-841e-831b32f20537\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-qmbs6"
Oct 30 18:42:23.508759 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.508734    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k6x5\" (UniqueName: \"kubernetes.io/projected/3e933bb3-b9ba-4dff-841e-831b32f20537-kube-api-access-6k6x5\") pod \"payload-pre-processing-7b98bb678c-qmbs6\" (UID: \"3e933bb3-b9ba-4dff-841e-831b32f20537\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-qmbs6"
Oct 30 18:42:23.522822 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.522786    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-7kxzm"]
Oct 30 18:42:23.526318 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.526297    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5f78687d5b-7kxzm"
Oct 30 18:42:23.565982 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.565916    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-7kxzm"]
Oct 30 18:42:23.609410 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.609384    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k6x5\" (UniqueName: \"kubernetes.io/projected/3e933bb3-b9ba-4dff-841e-831b32f20537-kube-api-access-6k6x5\") pod \"payload-pre-processing-7b98bb678c-qmbs6\" (UID: \"3e933bb3-b9ba-4dff-841e-831b32f20537\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-qmbs6"
Oct 30 18:42:23.609566 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.609421    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e933bb3-b9ba-4dff-841e-831b32f20537-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-qmbs6\" (UID: \"3e933bb3-b9ba-4dff-841e-831b32f20537\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-qmbs6"
Oct 30 18:42:23.609566 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.609474    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88b8cb4c-656b-44e4-8632-0acff65c58ad-plugins-config-volume\") pod \"payload-processing-5f78687d5b-7kxzm\" (UID: \"88b8cb4c-656b-44e4-8632-0acff65c58ad\") " pod="openshift-ingress/payload-processing-5f78687d5b-7kxzm"
Oct 30 18:42:23.609566 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.609506    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhj6g\" (UniqueName: \"kubernetes.io/projected/88b8cb4c-656b-44e4-8632-0acff65c58ad-kube-api-access-xhj6g\") pod \"payload-processing-5f78687d5b-7kxzm\" (UID: \"88b8cb4c-656b-44e4-8632-0acff65c58ad\") " pod="openshift-ingress/payload-processing-5f78687d5b-7kxzm"
Oct 30 18:42:23.610199 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.610182    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e933bb3-b9ba-4dff-841e-831b32f20537-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-qmbs6\" (UID: \"3e933bb3-b9ba-4dff-841e-831b32f20537\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-qmbs6"
Oct 30 18:42:23.670703 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.670631    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k6x5\" (UniqueName: \"kubernetes.io/projected/3e933bb3-b9ba-4dff-841e-831b32f20537-kube-api-access-6k6x5\") pod \"payload-pre-processing-7b98bb678c-qmbs6\" (UID: \"3e933bb3-b9ba-4dff-841e-831b32f20537\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-qmbs6"
Oct 30 18:42:23.683496 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.683471    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-qmbs6"
Oct 30 18:42:23.710718 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.710690    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhj6g\" (UniqueName: \"kubernetes.io/projected/88b8cb4c-656b-44e4-8632-0acff65c58ad-kube-api-access-xhj6g\") pod \"payload-processing-5f78687d5b-7kxzm\" (UID: \"88b8cb4c-656b-44e4-8632-0acff65c58ad\") " pod="openshift-ingress/payload-processing-5f78687d5b-7kxzm"
Oct 30 18:42:23.710828 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.710788    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88b8cb4c-656b-44e4-8632-0acff65c58ad-plugins-config-volume\") pod \"payload-processing-5f78687d5b-7kxzm\" (UID: \"88b8cb4c-656b-44e4-8632-0acff65c58ad\") " pod="openshift-ingress/payload-processing-5f78687d5b-7kxzm"
Oct 30 18:42:23.711458 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.711435    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88b8cb4c-656b-44e4-8632-0acff65c58ad-plugins-config-volume\") pod \"payload-processing-5f78687d5b-7kxzm\" (UID: \"88b8cb4c-656b-44e4-8632-0acff65c58ad\") " pod="openshift-ingress/payload-processing-5f78687d5b-7kxzm"
Oct 30 18:42:23.771553 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.771525    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhj6g\" (UniqueName: \"kubernetes.io/projected/88b8cb4c-656b-44e4-8632-0acff65c58ad-kube-api-access-xhj6g\") pod \"payload-processing-5f78687d5b-7kxzm\" (UID: \"88b8cb4c-656b-44e4-8632-0acff65c58ad\") " pod="openshift-ingress/payload-processing-5f78687d5b-7kxzm"
Oct 30 18:42:23.839231 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:23.839203    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5f78687d5b-7kxzm"
Oct 30 18:42:24.053682 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:24.053659    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-qmbs6"]
Oct 30 18:42:24.056312 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:42:24.056289    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e933bb3_b9ba_4dff_841e_831b32f20537.slice/crio-33bc80a37d2909acaf95833ff24f50b1d547cd625742ef0dda36ecb06228f16d WatchSource:0}: Error finding container 33bc80a37d2909acaf95833ff24f50b1d547cd625742ef0dda36ecb06228f16d: Status 404 returned error can't find the container with id 33bc80a37d2909acaf95833ff24f50b1d547cd625742ef0dda36ecb06228f16d
Oct 30 18:42:24.211913 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:24.211885    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-7kxzm"]
Oct 30 18:42:24.214869 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:42:24.214845    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88b8cb4c_656b_44e4_8632_0acff65c58ad.slice/crio-f47eae00478c6a4c44bce5d754ad1246add3cd7c021f7dfddaaf637216e6a1b1 WatchSource:0}: Error finding container f47eae00478c6a4c44bce5d754ad1246add3cd7c021f7dfddaaf637216e6a1b1: Status 404 returned error can't find the container with id f47eae00478c6a4c44bce5d754ad1246add3cd7c021f7dfddaaf637216e6a1b1
Oct 30 18:42:24.519077 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:24.518982    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5f78687d5b-7kxzm" event={"ID":"88b8cb4c-656b-44e4-8632-0acff65c58ad","Type":"ContainerStarted","Data":"f47eae00478c6a4c44bce5d754ad1246add3cd7c021f7dfddaaf637216e6a1b1"}
Oct 30 18:42:24.520066 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:24.520047    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-qmbs6" event={"ID":"3e933bb3-b9ba-4dff-841e-831b32f20537","Type":"ContainerStarted","Data":"33bc80a37d2909acaf95833ff24f50b1d547cd625742ef0dda36ecb06228f16d"}
Oct 30 18:42:27.538018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:27.537897    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5f78687d5b-7kxzm" event={"ID":"88b8cb4c-656b-44e4-8632-0acff65c58ad","Type":"ContainerStarted","Data":"9461d55ace1fecf249d9df1ac9fb0fdded2babd43775df97886ac317d99b010d"}
Oct 30 18:42:27.538425 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:27.538136    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5f78687d5b-7kxzm"
Oct 30 18:42:27.539394 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:27.539373    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-qmbs6" event={"ID":"3e933bb3-b9ba-4dff-841e-831b32f20537","Type":"ContainerStarted","Data":"c6c699b2d168d09142aebcd0270ddd17498c36e3054ecdbeb8dc5b8c90dca2a0"}
Oct 30 18:42:27.539490 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:27.539429    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-qmbs6"
Oct 30 18:42:27.653900 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:27.653851    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5f78687d5b-7kxzm" podStartSLOduration=1.786292593 podStartE2EDuration="4.653836802s" totalImagesPullingTime="2.867544209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:27.652337506 +0000 UTC m=+468.364086202" watchObservedRunningTime="2026-10-30 18:42:27.653836802 +0000 UTC m=+468.365585496"
Oct 30 18:42:27.767120 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:27.767066    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-qmbs6" podStartSLOduration=1.699298911 podStartE2EDuration="4.767051669s" totalImagesPullingTime="3.067752758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:27.766573932 +0000 UTC m=+468.478322629" watchObservedRunningTime="2026-10-30 18:42:27.767051669 +0000 UTC m=+468.478800366"
Oct 30 18:42:31.507162 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:31.507133    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-7d85675f98-2qfm6"
Oct 30 18:42:31.893521 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:31.893487    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-84cc5ff9cd-7vt6f"]
Oct 30 18:42:31.897185 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:31.897162    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-84cc5ff9cd-7vt6f"
Oct 30 18:42:31.956440 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:31.956403    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-84cc5ff9cd-7vt6f"]
Oct 30 18:42:32.087679 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:32.087641    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3c7887d-5d2f-47a9-8808-91251d470724-metrics-certs\") pod \"ai-gateway-controller-84cc5ff9cd-7vt6f\" (UID: \"f3c7887d-5d2f-47a9-8808-91251d470724\") " pod="opendatahub/ai-gateway-controller-84cc5ff9cd-7vt6f"
Oct 30 18:42:32.087859 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:32.087711    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hz5m\" (UniqueName: \"kubernetes.io/projected/f3c7887d-5d2f-47a9-8808-91251d470724-kube-api-access-7hz5m\") pod \"ai-gateway-controller-84cc5ff9cd-7vt6f\" (UID: \"f3c7887d-5d2f-47a9-8808-91251d470724\") " pod="opendatahub/ai-gateway-controller-84cc5ff9cd-7vt6f"
Oct 30 18:42:32.188749 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:32.188657    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hz5m\" (UniqueName: \"kubernetes.io/projected/f3c7887d-5d2f-47a9-8808-91251d470724-kube-api-access-7hz5m\") pod \"ai-gateway-controller-84cc5ff9cd-7vt6f\" (UID: \"f3c7887d-5d2f-47a9-8808-91251d470724\") " pod="opendatahub/ai-gateway-controller-84cc5ff9cd-7vt6f"
Oct 30 18:42:32.188749 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:32.188728    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3c7887d-5d2f-47a9-8808-91251d470724-metrics-certs\") pod \"ai-gateway-controller-84cc5ff9cd-7vt6f\" (UID: \"f3c7887d-5d2f-47a9-8808-91251d470724\") " pod="opendatahub/ai-gateway-controller-84cc5ff9cd-7vt6f"
Oct 30 18:42:32.191359 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:32.191337    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3c7887d-5d2f-47a9-8808-91251d470724-metrics-certs\") pod \"ai-gateway-controller-84cc5ff9cd-7vt6f\" (UID: \"f3c7887d-5d2f-47a9-8808-91251d470724\") " pod="opendatahub/ai-gateway-controller-84cc5ff9cd-7vt6f"
Oct 30 18:42:32.250096 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:32.250058    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hz5m\" (UniqueName: \"kubernetes.io/projected/f3c7887d-5d2f-47a9-8808-91251d470724-kube-api-access-7hz5m\") pod \"ai-gateway-controller-84cc5ff9cd-7vt6f\" (UID: \"f3c7887d-5d2f-47a9-8808-91251d470724\") " pod="opendatahub/ai-gateway-controller-84cc5ff9cd-7vt6f"
Oct 30 18:42:32.511882 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:32.511803    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-84cc5ff9cd-7vt6f"
Oct 30 18:42:32.883306 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:32.883270    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-84cc5ff9cd-7vt6f"]
Oct 30 18:42:32.886265 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:42:32.886240    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3c7887d_5d2f_47a9_8808_91251d470724.slice/crio-03a3654352776e3e0c36bd552f44c9a3f36981331ec2f96ea6667f986f93a853 WatchSource:0}: Error finding container 03a3654352776e3e0c36bd552f44c9a3f36981331ec2f96ea6667f986f93a853: Status 404 returned error can't find the container with id 03a3654352776e3e0c36bd552f44c9a3f36981331ec2f96ea6667f986f93a853
Oct 30 18:42:32.915922 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:32.915885    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:33.565888 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:33.565854    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-84cc5ff9cd-7vt6f" event={"ID":"f3c7887d-5d2f-47a9-8808-91251d470724","Type":"ContainerStarted","Data":"03a3654352776e3e0c36bd552f44c9a3f36981331ec2f96ea6667f986f93a853"}
Oct 30 18:42:34.571772 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:34.571735    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-84cc5ff9cd-7vt6f" event={"ID":"f3c7887d-5d2f-47a9-8808-91251d470724","Type":"ContainerStarted","Data":"012f21b8bfa1f5796fb67812407dbd50ead06f8431eb74d55e40ee3f856982d5"}
Oct 30 18:42:34.572307 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:34.571858    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-84cc5ff9cd-7vt6f"
Oct 30 18:42:34.689392 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:34.689340    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-84cc5ff9cd-7vt6f" podStartSLOduration=2.8039091430000003 podStartE2EDuration="3.689321742s" totalImagesPullingTime="885.412599ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:34.688161914 +0000 UTC m=+475.399910613" watchObservedRunningTime="2026-10-30 18:42:34.689321742 +0000 UTC m=+475.401070439"
Oct 30 18:42:38.202697 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:42:38.202656    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:42:38.546988 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:38.546889    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5f78687d5b-7kxzm"
Oct 30 18:42:38.546988 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:38.546981    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-qmbs6"
Oct 30 18:42:45.578712 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:45.578676    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-84cc5ff9cd-7vt6f"
Oct 30 18:42:45.747636 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:45.747602    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-2qfm6"]
Oct 30 18:42:45.747897 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:45.747844    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-7d85675f98-2qfm6" podUID="69426d3b-657a-4988-9dde-7dce4499e30d" containerName="manager" containerID="cri-o://5c246b4d0bf4e56eb983b148787620a2ffbc33bdfb2e10bd9b4d33f860de2f0c" gracePeriod=30
Oct 30 18:42:45.994444 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:45.994422    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-2qfm6"
Oct 30 18:42:46.114106 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:46.114025    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/69426d3b-657a-4988-9dde-7dce4499e30d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/69426d3b-657a-4988-9dde-7dce4499e30d-metrics-certs\") pod \"69426d3b-657a-4988-9dde-7dce4499e30d\" (UID: \"69426d3b-657a-4988-9dde-7dce4499e30d\") "
Oct 30 18:42:46.114265 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:46.114131    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/69426d3b-657a-4988-9dde-7dce4499e30d-kube-api-access-fcjxg\" (UniqueName: \"kubernetes.io/projected/69426d3b-657a-4988-9dde-7dce4499e30d-kube-api-access-fcjxg\") pod \"69426d3b-657a-4988-9dde-7dce4499e30d\" (UID: \"69426d3b-657a-4988-9dde-7dce4499e30d\") "
Oct 30 18:42:46.116474 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:46.116444    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69426d3b-657a-4988-9dde-7dce4499e30d-kube-api-access-fcjxg" pod "69426d3b-657a-4988-9dde-7dce4499e30d" (UID: "69426d3b-657a-4988-9dde-7dce4499e30d"). InnerVolumeSpecName "kube-api-access-fcjxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:42:46.116474 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:46.116460    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69426d3b-657a-4988-9dde-7dce4499e30d-metrics-certs" pod "69426d3b-657a-4988-9dde-7dce4499e30d" (UID: "69426d3b-657a-4988-9dde-7dce4499e30d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:42:46.215126 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:46.215091    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69426d3b-657a-4988-9dde-7dce4499e30d-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:42:46.215126 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:46.215124    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcjxg\" (UniqueName: \"kubernetes.io/projected/69426d3b-657a-4988-9dde-7dce4499e30d-kube-api-access-fcjxg\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:42:46.627683 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:46.627650    5820 generic.go:356] "Generic (PLEG): container finished" podID="69426d3b-657a-4988-9dde-7dce4499e30d" containerID="5c246b4d0bf4e56eb983b148787620a2ffbc33bdfb2e10bd9b4d33f860de2f0c" exitCode=0
Oct 30 18:42:46.628106 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:46.627703    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-2qfm6"
Oct 30 18:42:46.628106 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:46.627742    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-2qfm6" event={"ID":"69426d3b-657a-4988-9dde-7dce4499e30d","Type":"ContainerDied","Data":"5c246b4d0bf4e56eb983b148787620a2ffbc33bdfb2e10bd9b4d33f860de2f0c"}
Oct 30 18:42:46.628106 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:46.627800    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-2qfm6" event={"ID":"69426d3b-657a-4988-9dde-7dce4499e30d","Type":"ContainerDied","Data":"774105d54e10979d1e8b91c48ed31957589ad761e2f89740b6a1812176a21a54"}
Oct 30 18:42:46.628106 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:46.627830    5820 scope.go:122] "RemoveContainer" containerID="5c246b4d0bf4e56eb983b148787620a2ffbc33bdfb2e10bd9b4d33f860de2f0c"
Oct 30 18:42:46.637027 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:46.637007    5820 scope.go:122] "RemoveContainer" containerID="5c246b4d0bf4e56eb983b148787620a2ffbc33bdfb2e10bd9b4d33f860de2f0c"
Oct 30 18:42:46.637287 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:42:46.637269    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c246b4d0bf4e56eb983b148787620a2ffbc33bdfb2e10bd9b4d33f860de2f0c\": container with ID starting with 5c246b4d0bf4e56eb983b148787620a2ffbc33bdfb2e10bd9b4d33f860de2f0c not found: ID does not exist" containerID="5c246b4d0bf4e56eb983b148787620a2ffbc33bdfb2e10bd9b4d33f860de2f0c"
Oct 30 18:42:46.637335 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:46.637296    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c246b4d0bf4e56eb983b148787620a2ffbc33bdfb2e10bd9b4d33f860de2f0c"} err="failed to get container status \"5c246b4d0bf4e56eb983b148787620a2ffbc33bdfb2e10bd9b4d33f860de2f0c\": rpc error: code = NotFound desc = could not find container \"5c246b4d0bf4e56eb983b148787620a2ffbc33bdfb2e10bd9b4d33f860de2f0c\": container with ID starting with 5c246b4d0bf4e56eb983b148787620a2ffbc33bdfb2e10bd9b4d33f860de2f0c not found: ID does not exist"
Oct 30 18:42:46.810335 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:46.810306    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-2qfm6"]
Oct 30 18:42:46.859437 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:46.859405    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-2qfm6"]
Oct 30 18:42:47.435754 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:47.435720    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="69426d3b-657a-4988-9dde-7dce4499e30d" path="/var/lib/kubelet/pods/69426d3b-657a-4988-9dde-7dce4499e30d/volumes"
Oct 30 18:42:47.691262 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:47.691181    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:48.286031 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:48.285995    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:48.886671 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:48.886631    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:49.777285 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:49.777250    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:50.004003 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:50.003960    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:55.875119 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:55.875078    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:56.760477 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:56.760445    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"]
Oct 30 18:42:56.760856 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:56.760844    5820 cpu_manager.go:414] "containerMap: removing container" podUID="69426d3b-657a-4988-9dde-7dce4499e30d" containerName="manager"
Oct 30 18:42:56.760908 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:56.760858    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="69426d3b-657a-4988-9dde-7dce4499e30d" containerName="manager"
Oct 30 18:42:56.760962 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:56.760937    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="69426d3b-657a-4988-9dde-7dce4499e30d" containerName="manager"
Oct 30 18:42:56.766239 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:56.766213    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:42:56.819042 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:56.819011    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 18:42:56.868707 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:56.868676    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Oct 30 18:42:56.870222 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:56.870196    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"]
Oct 30 18:42:56.872993 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:56.872979    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 18:42:56.873064 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:56.873050    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-s7879\""
Oct 30 18:42:56.877968 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:56.877919    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:56.910888 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:56.910863    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa596309-c2e8-4ba3-bf40-02fedf140bb5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8\" (UID: \"aa596309-c2e8-4ba3-bf40-02fedf140bb5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:42:56.911058 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:56.910897    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa596309-c2e8-4ba3-bf40-02fedf140bb5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8\" (UID: \"aa596309-c2e8-4ba3-bf40-02fedf140bb5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:42:56.911058 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:56.910930    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa596309-c2e8-4ba3-bf40-02fedf140bb5-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8\" (UID: \"aa596309-c2e8-4ba3-bf40-02fedf140bb5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:42:56.911146 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:56.911069    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q5k7\" (UniqueName: \"kubernetes.io/projected/aa596309-c2e8-4ba3-bf40-02fedf140bb5-kube-api-access-8q5k7\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8\" (UID: \"aa596309-c2e8-4ba3-bf40-02fedf140bb5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:42:56.911146 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:56.911124    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa596309-c2e8-4ba3-bf40-02fedf140bb5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8\" (UID: \"aa596309-c2e8-4ba3-bf40-02fedf140bb5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:42:56.911218 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:56.911168    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa596309-c2e8-4ba3-bf40-02fedf140bb5-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8\" (UID: \"aa596309-c2e8-4ba3-bf40-02fedf140bb5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:42:57.012053 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:57.011974    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q5k7\" (UniqueName: \"kubernetes.io/projected/aa596309-c2e8-4ba3-bf40-02fedf140bb5-kube-api-access-8q5k7\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8\" (UID: \"aa596309-c2e8-4ba3-bf40-02fedf140bb5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:42:57.012053 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:57.012042    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa596309-c2e8-4ba3-bf40-02fedf140bb5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8\" (UID: \"aa596309-c2e8-4ba3-bf40-02fedf140bb5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:42:57.012265 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:57.012062    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa596309-c2e8-4ba3-bf40-02fedf140bb5-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8\" (UID: \"aa596309-c2e8-4ba3-bf40-02fedf140bb5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:42:57.012265 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:57.012096    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa596309-c2e8-4ba3-bf40-02fedf140bb5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8\" (UID: \"aa596309-c2e8-4ba3-bf40-02fedf140bb5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:42:57.012265 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:57.012115    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa596309-c2e8-4ba3-bf40-02fedf140bb5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8\" (UID: \"aa596309-c2e8-4ba3-bf40-02fedf140bb5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:42:57.012265 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:57.012242    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa596309-c2e8-4ba3-bf40-02fedf140bb5-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8\" (UID: \"aa596309-c2e8-4ba3-bf40-02fedf140bb5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:42:57.012474 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:57.012459    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa596309-c2e8-4ba3-bf40-02fedf140bb5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8\" (UID: \"aa596309-c2e8-4ba3-bf40-02fedf140bb5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:42:57.012548 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:57.012523    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa596309-c2e8-4ba3-bf40-02fedf140bb5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8\" (UID: \"aa596309-c2e8-4ba3-bf40-02fedf140bb5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:42:57.012610 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:57.012576    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa596309-c2e8-4ba3-bf40-02fedf140bb5-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8\" (UID: \"aa596309-c2e8-4ba3-bf40-02fedf140bb5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:42:57.014581 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:57.014564    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa596309-c2e8-4ba3-bf40-02fedf140bb5-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8\" (UID: \"aa596309-c2e8-4ba3-bf40-02fedf140bb5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:42:57.014681 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:57.014665    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa596309-c2e8-4ba3-bf40-02fedf140bb5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8\" (UID: \"aa596309-c2e8-4ba3-bf40-02fedf140bb5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:42:57.087108 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:57.087076    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q5k7\" (UniqueName: \"kubernetes.io/projected/aa596309-c2e8-4ba3-bf40-02fedf140bb5-kube-api-access-8q5k7\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8\" (UID: \"aa596309-c2e8-4ba3-bf40-02fedf140bb5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:42:57.379349 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:57.379318    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:42:57.777478 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:57.777400    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"]
Oct 30 18:42:57.781033 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:42:57.781008    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa596309_c2e8_4ba3_bf40_02fedf140bb5.slice/crio-62057f561b9ad3c38bb89f95ac2c9774f58790558cbc86d3c572d8fcc09be607 WatchSource:0}: Error finding container 62057f561b9ad3c38bb89f95ac2c9774f58790558cbc86d3c572d8fcc09be607: Status 404 returned error can't find the container with id 62057f561b9ad3c38bb89f95ac2c9774f58790558cbc86d3c572d8fcc09be607
Oct 30 18:42:57.782277 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:57.782261    5820 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:42:58.696354 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:42:58.696307    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8" event={"ID":"aa596309-c2e8-4ba3-bf40-02fedf140bb5","Type":"ContainerStarted","Data":"62057f561b9ad3c38bb89f95ac2c9774f58790558cbc86d3c572d8fcc09be607"}
Oct 30 18:43:00.707242 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:00.707205    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8" event={"ID":"aa596309-c2e8-4ba3-bf40-02fedf140bb5","Type":"ContainerStarted","Data":"473b0a5054b26a31248f1d0c33e67b52347f090eaa42073925ecc172806dbda1"}
Oct 30 18:43:00.825817 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:00.825765    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8" podStartSLOduration=2.893393947 podStartE2EDuration="4.825750578s" totalImagesPullingTime="1.932356631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:43:00.824765326 +0000 UTC m=+501.536514023" watchObservedRunningTime="2026-10-30 18:43:00.825750578 +0000 UTC m=+501.537499275"
Oct 30 18:43:01.380234 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:01.380202    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:43:03.391129 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.391096    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"]
Oct 30 18:43:03.398042 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.398017    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:03.454916 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.454889    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"]
Oct 30 18:43:03.470306 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.470275    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e273ced8-b9e7-479d-b9d5-33101d01e532-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bjq4x\" (UID: \"e273ced8-b9e7-479d-b9d5-33101d01e532\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:03.470474 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.470321    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e273ced8-b9e7-479d-b9d5-33101d01e532-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bjq4x\" (UID: \"e273ced8-b9e7-479d-b9d5-33101d01e532\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:03.470474 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.470341    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmc8c\" (UniqueName: \"kubernetes.io/projected/e273ced8-b9e7-479d-b9d5-33101d01e532-kube-api-access-fmc8c\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bjq4x\" (UID: \"e273ced8-b9e7-479d-b9d5-33101d01e532\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:03.470591 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.470492    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e273ced8-b9e7-479d-b9d5-33101d01e532-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bjq4x\" (UID: \"e273ced8-b9e7-479d-b9d5-33101d01e532\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:03.470591 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.470543    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e273ced8-b9e7-479d-b9d5-33101d01e532-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bjq4x\" (UID: \"e273ced8-b9e7-479d-b9d5-33101d01e532\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:03.470654 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.470628    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e273ced8-b9e7-479d-b9d5-33101d01e532-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bjq4x\" (UID: \"e273ced8-b9e7-479d-b9d5-33101d01e532\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:03.502247 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.502221    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Oct 30 18:43:03.523163 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.523135    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:03.571930 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.571899    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e273ced8-b9e7-479d-b9d5-33101d01e532-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bjq4x\" (UID: \"e273ced8-b9e7-479d-b9d5-33101d01e532\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:03.572131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.571990    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e273ced8-b9e7-479d-b9d5-33101d01e532-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bjq4x\" (UID: \"e273ced8-b9e7-479d-b9d5-33101d01e532\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:03.572131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.572020    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmc8c\" (UniqueName: \"kubernetes.io/projected/e273ced8-b9e7-479d-b9d5-33101d01e532-kube-api-access-fmc8c\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bjq4x\" (UID: \"e273ced8-b9e7-479d-b9d5-33101d01e532\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:03.572131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.572067    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e273ced8-b9e7-479d-b9d5-33101d01e532-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bjq4x\" (UID: \"e273ced8-b9e7-479d-b9d5-33101d01e532\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:03.572131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.572120    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e273ced8-b9e7-479d-b9d5-33101d01e532-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bjq4x\" (UID: \"e273ced8-b9e7-479d-b9d5-33101d01e532\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:03.572460 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.572155    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e273ced8-b9e7-479d-b9d5-33101d01e532-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bjq4x\" (UID: \"e273ced8-b9e7-479d-b9d5-33101d01e532\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:03.572460 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.572357    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e273ced8-b9e7-479d-b9d5-33101d01e532-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bjq4x\" (UID: \"e273ced8-b9e7-479d-b9d5-33101d01e532\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:03.572580 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.572541    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e273ced8-b9e7-479d-b9d5-33101d01e532-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bjq4x\" (UID: \"e273ced8-b9e7-479d-b9d5-33101d01e532\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:03.572751 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.572732    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e273ced8-b9e7-479d-b9d5-33101d01e532-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bjq4x\" (UID: \"e273ced8-b9e7-479d-b9d5-33101d01e532\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:03.574644 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.574619    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e273ced8-b9e7-479d-b9d5-33101d01e532-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bjq4x\" (UID: \"e273ced8-b9e7-479d-b9d5-33101d01e532\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:03.574864 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.574849    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e273ced8-b9e7-479d-b9d5-33101d01e532-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bjq4x\" (UID: \"e273ced8-b9e7-479d-b9d5-33101d01e532\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:03.633540 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.633499    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmc8c\" (UniqueName: \"kubernetes.io/projected/e273ced8-b9e7-479d-b9d5-33101d01e532-kube-api-access-fmc8c\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bjq4x\" (UID: \"e273ced8-b9e7-479d-b9d5-33101d01e532\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:03.711985 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:03.711880    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:04.095546 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:04.095427    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"]
Oct 30 18:43:04.098142 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:43:04.098109    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode273ced8_b9e7_479d_b9d5_33101d01e532.slice/crio-2821bdefb65fd9db1b940aab979baf98ef26ac4480e89965eafb3b4fac6ec909 WatchSource:0}: Error finding container 2821bdefb65fd9db1b940aab979baf98ef26ac4480e89965eafb3b4fac6ec909: Status 404 returned error can't find the container with id 2821bdefb65fd9db1b940aab979baf98ef26ac4480e89965eafb3b4fac6ec909
Oct 30 18:43:04.727343 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:04.727309    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x" event={"ID":"e273ced8-b9e7-479d-b9d5-33101d01e532","Type":"ContainerStarted","Data":"2821bdefb65fd9db1b940aab979baf98ef26ac4480e89965eafb3b4fac6ec909"}
Oct 30 18:43:05.732900 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:05.732859    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x" event={"ID":"e273ced8-b9e7-479d-b9d5-33101d01e532","Type":"ContainerStarted","Data":"b6e39ea2fea46dad1f1de5d60675cb6fec846df62518bf73ec4b8eed2af49342"}
Oct 30 18:43:05.852237 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:05.852163    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x" podStartSLOduration=2.270710463 podStartE2EDuration="2.852141207s" totalImagesPullingTime="581.430744ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:43:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:43:05.851049845 +0000 UTC m=+506.562798540" watchObservedRunningTime="2026-10-30 18:43:05.852141207 +0000 UTC m=+506.563889906"
Oct 30 18:43:06.712231 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:06.712196    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:06.842600 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:06.842566    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:07.380145 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:07.380105    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:43:07.392604 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:07.392573    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:43:07.754201 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:07.754127    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8"
Oct 30 18:43:08.202658 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:43:08.202619    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:43:11.738215 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:11.738168    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:13.712965 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:13.712924    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:13.725313 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:13.725286    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:13.781107 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:13.781080    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bjq4x"
Oct 30 18:43:16.585662 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:16.585624    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:21.064032 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.063997    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-k5crh"]
Oct 30 18:43:21.067175 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.067155    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:21.128521 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.128497    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\""
Oct 30 18:43:21.132998 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.132973    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjztd\" (UniqueName: \"kubernetes.io/projected/971c8953-1bc9-4276-9767-5a99d6a215ba-kube-api-access-qjztd\") pod \"echo-simulator-kserve-64b6f8f96c-k5crh\" (UID: \"971c8953-1bc9-4276-9767-5a99d6a215ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:21.133126 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.133015    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/971c8953-1bc9-4276-9767-5a99d6a215ba-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-k5crh\" (UID: \"971c8953-1bc9-4276-9767-5a99d6a215ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:21.133182 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.133159    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/971c8953-1bc9-4276-9767-5a99d6a215ba-home\") pod \"echo-simulator-kserve-64b6f8f96c-k5crh\" (UID: \"971c8953-1bc9-4276-9767-5a99d6a215ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:21.133232 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.133212    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/971c8953-1bc9-4276-9767-5a99d6a215ba-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-k5crh\" (UID: \"971c8953-1bc9-4276-9767-5a99d6a215ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:21.133270 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.133235    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/971c8953-1bc9-4276-9767-5a99d6a215ba-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-k5crh\" (UID: \"971c8953-1bc9-4276-9767-5a99d6a215ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:21.133270 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.133260    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/971c8953-1bc9-4276-9767-5a99d6a215ba-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-k5crh\" (UID: \"971c8953-1bc9-4276-9767-5a99d6a215ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:21.136335 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.136305    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-k5crh"]
Oct 30 18:43:21.233713 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.233682    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/971c8953-1bc9-4276-9767-5a99d6a215ba-home\") pod \"echo-simulator-kserve-64b6f8f96c-k5crh\" (UID: \"971c8953-1bc9-4276-9767-5a99d6a215ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:21.233713 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.233719    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/971c8953-1bc9-4276-9767-5a99d6a215ba-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-k5crh\" (UID: \"971c8953-1bc9-4276-9767-5a99d6a215ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:21.233924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.233774    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/971c8953-1bc9-4276-9767-5a99d6a215ba-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-k5crh\" (UID: \"971c8953-1bc9-4276-9767-5a99d6a215ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:21.233924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.233840    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/971c8953-1bc9-4276-9767-5a99d6a215ba-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-k5crh\" (UID: \"971c8953-1bc9-4276-9767-5a99d6a215ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:21.234075 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.233940    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjztd\" (UniqueName: \"kubernetes.io/projected/971c8953-1bc9-4276-9767-5a99d6a215ba-kube-api-access-qjztd\") pod \"echo-simulator-kserve-64b6f8f96c-k5crh\" (UID: \"971c8953-1bc9-4276-9767-5a99d6a215ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:21.234075 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.234012    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/971c8953-1bc9-4276-9767-5a99d6a215ba-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-k5crh\" (UID: \"971c8953-1bc9-4276-9767-5a99d6a215ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:21.234184 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.234163    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/971c8953-1bc9-4276-9767-5a99d6a215ba-home\") pod \"echo-simulator-kserve-64b6f8f96c-k5crh\" (UID: \"971c8953-1bc9-4276-9767-5a99d6a215ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:21.234255 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.234222    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/971c8953-1bc9-4276-9767-5a99d6a215ba-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-k5crh\" (UID: \"971c8953-1bc9-4276-9767-5a99d6a215ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:21.234315 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.234296    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/971c8953-1bc9-4276-9767-5a99d6a215ba-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-k5crh\" (UID: \"971c8953-1bc9-4276-9767-5a99d6a215ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:21.236338 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.236307    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/971c8953-1bc9-4276-9767-5a99d6a215ba-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-k5crh\" (UID: \"971c8953-1bc9-4276-9767-5a99d6a215ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:21.236461 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.236443    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/971c8953-1bc9-4276-9767-5a99d6a215ba-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-k5crh\" (UID: \"971c8953-1bc9-4276-9767-5a99d6a215ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:21.239370 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.239344    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:21.295424 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.295393    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjztd\" (UniqueName: \"kubernetes.io/projected/971c8953-1bc9-4276-9767-5a99d6a215ba-kube-api-access-qjztd\") pod \"echo-simulator-kserve-64b6f8f96c-k5crh\" (UID: \"971c8953-1bc9-4276-9767-5a99d6a215ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:21.380163 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.380073    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:21.761078 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.760996    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-k5crh"]
Oct 30 18:43:21.763880 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:43:21.763855    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod971c8953_1bc9_4276_9767_5a99d6a215ba.slice/crio-7429d265339e81069710eaa047a3b416c253ff67b794597f4d5aa071dae907ba WatchSource:0}: Error finding container 7429d265339e81069710eaa047a3b416c253ff67b794597f4d5aa071dae907ba: Status 404 returned error can't find the container with id 7429d265339e81069710eaa047a3b416c253ff67b794597f4d5aa071dae907ba
Oct 30 18:43:21.805166 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:21.805128    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh" event={"ID":"971c8953-1bc9-4276-9767-5a99d6a215ba","Type":"ContainerStarted","Data":"7429d265339e81069710eaa047a3b416c253ff67b794597f4d5aa071dae907ba"}
Oct 30 18:43:24.709851 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.709820    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"]
Oct 30 18:43:24.713563 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.713544    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:24.762893 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.762867    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a765d287-6e9e-408e-91b6-5a32ab049574-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t\" (UID: \"a765d287-6e9e-408e-91b6-5a32ab049574\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:24.763086 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.762901    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a765d287-6e9e-408e-91b6-5a32ab049574-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t\" (UID: \"a765d287-6e9e-408e-91b6-5a32ab049574\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:24.763086 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.762921    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvt6n\" (UniqueName: \"kubernetes.io/projected/a765d287-6e9e-408e-91b6-5a32ab049574-kube-api-access-dvt6n\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t\" (UID: \"a765d287-6e9e-408e-91b6-5a32ab049574\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:24.763086 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.763011    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a765d287-6e9e-408e-91b6-5a32ab049574-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t\" (UID: \"a765d287-6e9e-408e-91b6-5a32ab049574\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:24.763086 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.763075    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a765d287-6e9e-408e-91b6-5a32ab049574-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t\" (UID: \"a765d287-6e9e-408e-91b6-5a32ab049574\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:24.763287 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.763103    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a765d287-6e9e-408e-91b6-5a32ab049574-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t\" (UID: \"a765d287-6e9e-408e-91b6-5a32ab049574\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:24.815180 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.815152    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Oct 30 18:43:24.816677 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.816651    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"]
Oct 30 18:43:24.821610 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.821584    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh" event={"ID":"971c8953-1bc9-4276-9767-5a99d6a215ba","Type":"ContainerStarted","Data":"58190fdb6718e64fe849c2a9320f7ccf6708c520cbc60c0c5719f771afcd9d37"}
Oct 30 18:43:24.854736 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.854708    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:24.863983 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.863939    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a765d287-6e9e-408e-91b6-5a32ab049574-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t\" (UID: \"a765d287-6e9e-408e-91b6-5a32ab049574\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:24.864105 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.864002    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a765d287-6e9e-408e-91b6-5a32ab049574-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t\" (UID: \"a765d287-6e9e-408e-91b6-5a32ab049574\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:24.864105 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.864021    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvt6n\" (UniqueName: \"kubernetes.io/projected/a765d287-6e9e-408e-91b6-5a32ab049574-kube-api-access-dvt6n\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t\" (UID: \"a765d287-6e9e-408e-91b6-5a32ab049574\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:24.864105 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.864047    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a765d287-6e9e-408e-91b6-5a32ab049574-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t\" (UID: \"a765d287-6e9e-408e-91b6-5a32ab049574\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:24.864248 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.864120    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a765d287-6e9e-408e-91b6-5a32ab049574-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t\" (UID: \"a765d287-6e9e-408e-91b6-5a32ab049574\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:24.864248 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.864147    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a765d287-6e9e-408e-91b6-5a32ab049574-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t\" (UID: \"a765d287-6e9e-408e-91b6-5a32ab049574\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:24.864456 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.864429    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a765d287-6e9e-408e-91b6-5a32ab049574-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t\" (UID: \"a765d287-6e9e-408e-91b6-5a32ab049574\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:24.864567 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.864486    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a765d287-6e9e-408e-91b6-5a32ab049574-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t\" (UID: \"a765d287-6e9e-408e-91b6-5a32ab049574\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:24.864617 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.864573    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a765d287-6e9e-408e-91b6-5a32ab049574-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t\" (UID: \"a765d287-6e9e-408e-91b6-5a32ab049574\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:24.867095 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.867073    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a765d287-6e9e-408e-91b6-5a32ab049574-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t\" (UID: \"a765d287-6e9e-408e-91b6-5a32ab049574\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:24.867247 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.867231    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a765d287-6e9e-408e-91b6-5a32ab049574-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t\" (UID: \"a765d287-6e9e-408e-91b6-5a32ab049574\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:24.925299 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.925263    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvt6n\" (UniqueName: \"kubernetes.io/projected/a765d287-6e9e-408e-91b6-5a32ab049574-kube-api-access-dvt6n\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t\" (UID: \"a765d287-6e9e-408e-91b6-5a32ab049574\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:24.951135 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:24.951089    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh" podStartSLOduration=1.7262943069999999 podStartE2EDuration="3.951075125s" totalImagesPullingTime="2.224780818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:43:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:43:24.948680695 +0000 UTC m=+525.660429392" watchObservedRunningTime="2026-10-30 18:43:24.951075125 +0000 UTC m=+525.662823822"
Oct 30 18:43:25.026258 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:25.026174    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:25.380346 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:25.380293    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:25.402721 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:25.402694    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"]
Oct 30 18:43:25.406043 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:43:25.406014    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda765d287_6e9e_408e_91b6_5a32ab049574.slice/crio-47f351d800ad1ad46d80bbcf46c964fb4a74c811e90996435e971675f3945531 WatchSource:0}: Error finding container 47f351d800ad1ad46d80bbcf46c964fb4a74c811e90996435e971675f3945531: Status 404 returned error can't find the container with id 47f351d800ad1ad46d80bbcf46c964fb4a74c811e90996435e971675f3945531
Oct 30 18:43:25.827904 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:25.827871    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t" event={"ID":"a765d287-6e9e-408e-91b6-5a32ab049574","Type":"ContainerStarted","Data":"47f351d800ad1ad46d80bbcf46c964fb4a74c811e90996435e971675f3945531"}
Oct 30 18:43:26.834055 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:26.834018    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t" event={"ID":"a765d287-6e9e-408e-91b6-5a32ab049574","Type":"ContainerStarted","Data":"ee8490642fdb683a5b01dbe2156297f5c40ad69cf60373970dfd8a79a46785cd"}
Oct 30 18:43:26.953143 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:26.953089    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t" podStartSLOduration=2.348113549 podStartE2EDuration="2.953074877s" totalImagesPullingTime="604.961328ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:43:26.952378284 +0000 UTC m=+527.664126980" watchObservedRunningTime="2026-10-30 18:43:26.953074877 +0000 UTC m=+527.664823574"
Oct 30 18:43:27.026604 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:27.026572    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:27.931079 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:27.931046    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:31.380993 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:31.380943    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:31.393319 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:31.393298    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:31.868440 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:31.868411    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-k5crh"
Oct 30 18:43:35.026312 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:35.026275    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:35.038848 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:35.038824    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:35.887690 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:35.887662    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t"
Oct 30 18:43:35.993015 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:35.992985    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"]
Oct 30 18:43:35.998899 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:35.998884    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.051126 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.051090    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-s874n\""
Oct 30 18:43:36.065048 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.065023    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.065194 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.065061    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.065194 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.065082    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.065194 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.065103    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.065315 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.065223    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.065315 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.065266    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.065389 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.065336    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9pj5\" (UniqueName: \"kubernetes.io/projected/fb366d57-3daf-4ffc-a3e8-e771542ae33f-kube-api-access-g9pj5\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.065389 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.065371    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.065500 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.065405    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.083667 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.083637    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"]
Oct 30 18:43:36.166976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.166873    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9pj5\" (UniqueName: \"kubernetes.io/projected/fb366d57-3daf-4ffc-a3e8-e771542ae33f-kube-api-access-g9pj5\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.166976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.166916    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.167251 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.167075    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.167251 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.167210    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.167251 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.167244    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.167414 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.167273    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.167414 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.167308    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.167414 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.167348    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.167414 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.167376    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.167606 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.167420    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.167606 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.167547    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.167788 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.167763    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.167896 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.167854    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.168023 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.168007    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.169602 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.169582    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.169797 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.169775    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.232770 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.232739    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.232898 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.232867    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9pj5\" (UniqueName: \"kubernetes.io/projected/fb366d57-3daf-4ffc-a3e8-e771542ae33f-kube-api-access-g9pj5\") pod \"maas-default-gateway-openshift-default-66db998887-2r7zc\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.312514 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.312481    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:36.804435 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.804404    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"]
Oct 30 18:43:36.807850 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:43:36.807823    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb366d57_3daf_4ffc_a3e8_e771542ae33f.slice/crio-5921fc67f427b2c0b0b1e62e835d636fb5f7afcce738260fd752456b5e33dbbb WatchSource:0}: Error finding container 5921fc67f427b2c0b0b1e62e835d636fb5f7afcce738260fd752456b5e33dbbb: Status 404 returned error can't find the container with id 5921fc67f427b2c0b0b1e62e835d636fb5f7afcce738260fd752456b5e33dbbb
Oct 30 18:43:36.883172 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:36.883139    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc" event={"ID":"fb366d57-3daf-4ffc-a3e8-e771542ae33f","Type":"ContainerStarted","Data":"5921fc67f427b2c0b0b1e62e835d636fb5f7afcce738260fd752456b5e33dbbb"}
Oct 30 18:43:38.202424 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:43:38.202387    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:43:41.158699 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:41.158662    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:43:41.159038 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:41.158732    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:43:41.159038 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:41.158763    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:43:41.911375 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:41.911338    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc" event={"ID":"fb366d57-3daf-4ffc-a3e8-e771542ae33f","Type":"ContainerStarted","Data":"4e42101efa6917d64a3e2fb8e207fe9e4761431ee955388334536a7229bdfbad"}
Oct 30 18:43:42.313021 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:42.312989    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:42.314422 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:42.314394    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-2r7zc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 18:43:42.314536 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:42.314454    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc" podUID="fb366d57-3daf-4ffc-a3e8-e771542ae33f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 18:43:43.313003 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:43.312940    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-2r7zc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 18:43:43.313210 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:43.313063    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc" podUID="fb366d57-3daf-4ffc-a3e8-e771542ae33f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 18:43:44.313613 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:44.313577    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-2r7zc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 18:43:44.314036 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:44.313630    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc" podUID="fb366d57-3daf-4ffc-a3e8-e771542ae33f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 18:43:45.313401 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:45.313363    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-2r7zc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 18:43:45.313589 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:45.313425    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc" podUID="fb366d57-3daf-4ffc-a3e8-e771542ae33f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 18:43:46.313043 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:46.313006    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-2r7zc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 18:43:46.313413 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:46.313058    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc" podUID="fb366d57-3daf-4ffc-a3e8-e771542ae33f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 18:43:47.312854 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:47.312778    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-2r7zc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 18:43:47.312854 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:47.312841    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc" podUID="fb366d57-3daf-4ffc-a3e8-e771542ae33f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 18:43:48.313908 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:48.313861    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-2r7zc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 18:43:48.314417 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:48.313935    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc" podUID="fb366d57-3daf-4ffc-a3e8-e771542ae33f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 18:43:49.230634 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:49.230585    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:49.319099 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:49.319063    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:49.319592 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:49.319423    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:49.320294 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:49.320273    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 18:43:49.412708 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:49.412668    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:49.495357 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:43:49.495248    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc" podStartSLOduration=10.147143509 podStartE2EDuration="14.495234219s" totalImagesPullingTime="4.34809071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:43:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:43:42.060229727 +0000 UTC m=+542.771978425" watchObservedRunningTime="2026-10-30 18:43:49.495234219 +0000 UTC m=+550.206982915"
Oct 30 18:44:00.779965 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:00.779912    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:00.980697 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:00.980661    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:02.316105 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:02.316071    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:02.596982 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:02.596886    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:06.221217 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.221183    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"]
Oct 30 18:44:06.229347 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.229320    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.268015 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.267981    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.268157 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.268026    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.268157 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.268112    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.268306 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.268160    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.268306 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.268224    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb8c3600-6aad-46a7-86a2-90f9962f461b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.268306 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.268274    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.268414 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.268363    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.268414 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.268391    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx5x9\" (UniqueName: \"kubernetes.io/projected/cb8c3600-6aad-46a7-86a2-90f9962f461b-kube-api-access-hx5x9\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.268537 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.268503    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.369921 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.369888    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.369921 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.369921    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx5x9\" (UniqueName: \"kubernetes.io/projected/cb8c3600-6aad-46a7-86a2-90f9962f461b-kube-api-access-hx5x9\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.370140 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.369993    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.370140 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.370018    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.370140 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.370052    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.370140 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.370086    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.370140 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.370133    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.370399 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.370156    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb8c3600-6aad-46a7-86a2-90f9962f461b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.370399 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.370191    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.370399 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.370339    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.370554 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.370454    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.370554 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.370531    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.370791 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.370771    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.370883 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.370826    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb8c3600-6aad-46a7-86a2-90f9962f461b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.372465 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.372444    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.372647 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.372631    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.430333 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.430293    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"]
Oct 30 18:44:06.437983 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.434502    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx5x9\" (UniqueName: \"kubernetes.io/projected/cb8c3600-6aad-46a7-86a2-90f9962f461b-kube-api-access-hx5x9\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.437983 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.434617    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-mfkwp\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.544673 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.544590    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:06.974923 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.974795    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"]
Oct 30 18:44:06.977677 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:44:06.977653    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb8c3600_6aad_46a7_86a2_90f9962f461b.slice/crio-d0b177f161b165ff20f15b5cb6c4f1af302f9145f1e805e1d5ece73786b437d4 WatchSource:0}: Error finding container d0b177f161b165ff20f15b5cb6c4f1af302f9145f1e805e1d5ece73786b437d4: Status 404 returned error can't find the container with id d0b177f161b165ff20f15b5cb6c4f1af302f9145f1e805e1d5ece73786b437d4
Oct 30 18:44:06.980550 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.980521    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:44:06.980619 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.980588    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:44:06.980619 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:06.980615    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:44:07.032350 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:07.032322    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp" event={"ID":"cb8c3600-6aad-46a7-86a2-90f9962f461b","Type":"ContainerStarted","Data":"d0b177f161b165ff20f15b5cb6c4f1af302f9145f1e805e1d5ece73786b437d4"}
Oct 30 18:44:08.060965 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:08.060923    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp" event={"ID":"cb8c3600-6aad-46a7-86a2-90f9962f461b","Type":"ContainerStarted","Data":"a448d92f5523b201dbe4acb1da275c8a506473897d4d7dee9b2311ae78debf48"}
Oct 30 18:44:08.202606 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:44:08.202568    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:44:08.551032 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:08.550971    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:08.559115 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:08.559076    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-mfkwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 18:44:08.559273 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:08.559135    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp" podUID="cb8c3600-6aad-46a7-86a2-90f9962f461b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 18:44:09.554976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:09.554364    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-mfkwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 18:44:09.554976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:09.554430    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp" podUID="cb8c3600-6aad-46a7-86a2-90f9962f461b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 18:44:09.597827 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:09.597793    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:10.554970 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:10.554333    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-mfkwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 18:44:10.554970 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:10.554415    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp" podUID="cb8c3600-6aad-46a7-86a2-90f9962f461b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 18:44:11.552976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:11.549316    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-mfkwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 18:44:11.552976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:11.549379    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp" podUID="cb8c3600-6aad-46a7-86a2-90f9962f461b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 18:44:12.323359 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:12.323324    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:12.552974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:12.548327    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-mfkwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 18:44:12.552974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:12.548401    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp" podUID="cb8c3600-6aad-46a7-86a2-90f9962f461b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 18:44:13.544908 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:13.544880    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-mfkwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 18:44:13.545310 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:13.544933    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp" podUID="cb8c3600-6aad-46a7-86a2-90f9962f461b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 18:44:15.155968 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:15.155907    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:15.156455 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:15.156388    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:15.157665 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:15.157645    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 18:44:15.336214 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:15.336153    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp" podStartSLOduration=9.336138917 podStartE2EDuration="9.336138917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:08.241249084 +0000 UTC m=+568.952997785" watchObservedRunningTime="2026-10-30 18:44:15.336138917 +0000 UTC m=+576.047887614"
Oct 30 18:44:32.139688 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:32.139650    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:32.959765 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:32.959720    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:33.263796 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.263713    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:33.544383 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.544301    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:33.695365 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.695333    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"]
Oct 30 18:44:33.698843 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.698827    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.753187 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.753148    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-529e04e2-openshift-default-dockercfg-79dpk\""
Oct 30 18:44:33.805118 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.805089    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"]
Oct 30 18:44:33.844846 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.844809    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-data\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.845076 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.844892    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-workload-socket\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.845076 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.845003    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-workload-certs\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.845076 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.845045    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-envoy\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.845268 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.845085    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-credential-socket\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.845268 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.845121    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istiod-ca-cert\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.845268 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.845230    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-podinfo\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.845268 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.845264    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-token\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.845422 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.845340    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz6mn\" (UniqueName: \"kubernetes.io/projected/3a5376da-e0ec-4364-98ed-1079db8ac8f7-kube-api-access-gz6mn\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.850868 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.850830    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:33.946855 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.946816    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-podinfo\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.947086 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.946881    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-token\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.947086 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.946978    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz6mn\" (UniqueName: \"kubernetes.io/projected/3a5376da-e0ec-4364-98ed-1079db8ac8f7-kube-api-access-gz6mn\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.947086 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.947032    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-data\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.947086 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.947086    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-workload-socket\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.947336 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.947110    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-workload-certs\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.947336 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.947146    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-envoy\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.947336 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.947185    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-credential-socket\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.947336 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.947221    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istiod-ca-cert\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.948310 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.947735    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-workload-socket\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.948310 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.947991    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istiod-ca-cert\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.948512 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.948398    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-workload-certs\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.949759 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.949728    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-data\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.950751 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.950732    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-credential-socket\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.954986 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.953906    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-envoy\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:33.954986 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:33.954113    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-podinfo\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:34.040964 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.040919    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz6mn\" (UniqueName: \"kubernetes.io/projected/3a5376da-e0ec-4364-98ed-1079db8ac8f7-kube-api-access-gz6mn\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:34.042453 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.042432    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-token\") pod \"e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") " pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:34.312461 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.312428    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:34.487807 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.487775    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"]
Oct 30 18:44:34.491579 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.491558    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.545381 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.545354    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-16a49490-gw-openshift-default-dockercfg-r74kq\""
Oct 30 18:44:34.557400 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.557367    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"]
Oct 30 18:44:34.656906 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.656824    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-workload-socket\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.656906 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.656876    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-istio-envoy\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.657117 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.656984    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-istio-data\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.657117 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.657017    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/358d8765-c33a-4934-a833-2d6fe588780e-istiod-ca-cert\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.657117 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.657050    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/358d8765-c33a-4934-a833-2d6fe588780e-istio-token\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.657214 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.657134    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-credential-socket\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.657248 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.657232    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kncn8\" (UniqueName: \"kubernetes.io/projected/358d8765-c33a-4934-a833-2d6fe588780e-kube-api-access-kncn8\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.657282 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.657272    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/358d8765-c33a-4934-a833-2d6fe588780e-istio-podinfo\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.657317 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.657297    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-workload-certs\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.758366 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.758333    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kncn8\" (UniqueName: \"kubernetes.io/projected/358d8765-c33a-4934-a833-2d6fe588780e-kube-api-access-kncn8\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.758556 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.758385    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/358d8765-c33a-4934-a833-2d6fe588780e-istio-podinfo\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.758556 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.758421    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-workload-certs\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.758556 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.758476    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-workload-socket\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.758556 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.758513    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-istio-envoy\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.758556 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.758541    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-istio-data\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.758789 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.758578    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/358d8765-c33a-4934-a833-2d6fe588780e-istiod-ca-cert\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.758789 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.758619    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/358d8765-c33a-4934-a833-2d6fe588780e-istio-token\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.758789 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.758649    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-credential-socket\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.759260 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.759230    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-workload-certs\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.759390 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.759342    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-istio-data\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.759460 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.759420    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-credential-socket\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.759550 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.759517    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-workload-socket\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.759667 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.759648    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/358d8765-c33a-4934-a833-2d6fe588780e-istiod-ca-cert\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.761140 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.761120    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/358d8765-c33a-4934-a833-2d6fe588780e-istio-podinfo\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.761223 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.761167    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-istio-envoy\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.777903 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.777885    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"]
Oct 30 18:44:34.780396 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:44:34.780374    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a5376da_e0ec_4364_98ed_1079db8ac8f7.slice/crio-b201c252fe396d4be55fdccf5155261e08a4e5ddd28c2b62aba0d8a05e2bc664 WatchSource:0}: Error finding container b201c252fe396d4be55fdccf5155261e08a4e5ddd28c2b62aba0d8a05e2bc664: Status 404 returned error can't find the container with id b201c252fe396d4be55fdccf5155261e08a4e5ddd28c2b62aba0d8a05e2bc664
Oct 30 18:44:34.782589 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.782553    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:44:34.782687 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.782635    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:44:34.782687 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.782678    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:44:34.824657 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.824632    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kncn8\" (UniqueName: \"kubernetes.io/projected/358d8765-c33a-4934-a833-2d6fe588780e-kube-api-access-kncn8\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:34.824748 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:34.824658    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/358d8765-c33a-4934-a833-2d6fe588780e-istio-token\") pod \"e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") " pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:35.121102 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:35.121063    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:35.129846 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:35.129815    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:35.197009 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:35.196935    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h" event={"ID":"3a5376da-e0ec-4364-98ed-1079db8ac8f7","Type":"ContainerStarted","Data":"f77cdfc7ac1681d95860448512b5812dcd8f5b642413a8de751f4c0f9a5340d2"}
Oct 30 18:44:35.197154 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:35.197043    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h" event={"ID":"3a5376da-e0ec-4364-98ed-1079db8ac8f7","Type":"ContainerStarted","Data":"b201c252fe396d4be55fdccf5155261e08a4e5ddd28c2b62aba0d8a05e2bc664"}
Oct 30 18:44:35.312979 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:35.312925    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:35.318046 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:35.318027    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:35.319720 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:35.319370    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h" podStartSLOduration=2.319352718 podStartE2EDuration="2.319352718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:35.319081261 +0000 UTC m=+596.030829949" watchObservedRunningTime="2026-10-30 18:44:35.319352718 +0000 UTC m=+596.031101417"
Oct 30 18:44:35.383990 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:35.383906    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:35.553809 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:35.553772    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"]
Oct 30 18:44:35.556436 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:44:35.556404    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod358d8765_c33a_4934_a833_2d6fe588780e.slice/crio-8c28e028d88bb00f65c7c305a63d05578301d6af12659f3137c57a7bdadbe07d WatchSource:0}: Error finding container 8c28e028d88bb00f65c7c305a63d05578301d6af12659f3137c57a7bdadbe07d: Status 404 returned error can't find the container with id 8c28e028d88bb00f65c7c305a63d05578301d6af12659f3137c57a7bdadbe07d
Oct 30 18:44:35.558919 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:35.558889    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:44:35.559074 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:35.558971    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:44:35.559074 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:35.559006    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:44:36.210057 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:36.210013    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66" event={"ID":"358d8765-c33a-4934-a833-2d6fe588780e","Type":"ContainerStarted","Data":"cc3b8eb6adc7a7eba2e44c66e0d4eb6a97a340541c0eafd725094610f5de3c91"}
Oct 30 18:44:36.210273 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:36.210107    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66" event={"ID":"358d8765-c33a-4934-a833-2d6fe588780e","Type":"ContainerStarted","Data":"8c28e028d88bb00f65c7c305a63d05578301d6af12659f3137c57a7bdadbe07d"}
Oct 30 18:44:36.210273 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:36.210218    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:36.211150 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:36.211131    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:44:36.337425 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:36.337374    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66" podStartSLOduration=2.337359589 podStartE2EDuration="2.337359589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:36.335798992 +0000 UTC m=+597.047547689" watchObservedRunningTime="2026-10-30 18:44:36.337359589 +0000 UTC m=+597.049108336"
Oct 30 18:44:36.592639 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:36.592603    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:36.906160 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:36.906082    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:37.121710 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:37.121682    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:37.126605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:37.126584    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:37.215570 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:37.215494    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:37.216301 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:37.216284    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:44:37.653938 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:37.653903    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:37.865499 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:37.865356    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:38.205052 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:44:38.205018    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:44:38.434317 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:38.434286    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:38.650620 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:38.650593    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:40.890504 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:40.890469    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:41.094747 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:41.094711    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:41.306605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:41.306572    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:41.419271 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:41.419241    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr"]
Oct 30 18:44:41.424135 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:41.424111    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr"
Oct 30 18:44:41.476345 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:41.476320    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-529e04e2\""
Oct 30 18:44:41.517757 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:41.517719    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpvpt\" (UniqueName: \"kubernetes.io/projected/89fdf7cd-72bb-4e56-a826-2df992558d41-kube-api-access-dpvpt\") pod \"payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr\" (UID: \"89fdf7cd-72bb-4e56-a826-2df992558d41\") " pod="openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr"
Oct 30 18:44:41.517965 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:41.517930    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89fdf7cd-72bb-4e56-a826-2df992558d41-plugins-config-volume\") pod \"payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr\" (UID: \"89fdf7cd-72bb-4e56-a826-2df992558d41\") " pod="openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr"
Oct 30 18:44:41.525731 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:41.525712    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-529e04e2-dockercfg-wstvs\""
Oct 30 18:44:41.527541 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:41.527523    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr"]
Oct 30 18:44:41.618931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:41.618831    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpvpt\" (UniqueName: \"kubernetes.io/projected/89fdf7cd-72bb-4e56-a826-2df992558d41-kube-api-access-dpvpt\") pod \"payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr\" (UID: \"89fdf7cd-72bb-4e56-a826-2df992558d41\") " pod="openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr"
Oct 30 18:44:41.619137 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:41.619032    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89fdf7cd-72bb-4e56-a826-2df992558d41-plugins-config-volume\") pod \"payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr\" (UID: \"89fdf7cd-72bb-4e56-a826-2df992558d41\") " pod="openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr"
Oct 30 18:44:41.619727 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:41.619710    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89fdf7cd-72bb-4e56-a826-2df992558d41-plugins-config-volume\") pod \"payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr\" (UID: \"89fdf7cd-72bb-4e56-a826-2df992558d41\") " pod="openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr"
Oct 30 18:44:41.633683 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:41.633657    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:41.679869 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:41.679847    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpvpt\" (UniqueName: \"kubernetes.io/projected/89fdf7cd-72bb-4e56-a826-2df992558d41-kube-api-access-dpvpt\") pod \"payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr\" (UID: \"89fdf7cd-72bb-4e56-a826-2df992558d41\") " pod="openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr"
Oct 30 18:44:41.738069 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:41.738041    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr"
Oct 30 18:44:42.089390 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.089358    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"]
Oct 30 18:44:42.094179 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.094161    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.115569 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.115543    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr"]
Oct 30 18:44:42.123645 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.123562    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-token\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.123645 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.123624    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-workload-socket\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.123785 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.123653    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-data\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.123785 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.123693    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istiod-ca-cert\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.123785 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.123721    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-podinfo\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.123940 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.123769    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-credential-socket\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.123940 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.123831    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-envoy\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.123940 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.123872    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-workload-certs\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.124110 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.123979    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgt6b\" (UniqueName: \"kubernetes.io/projected/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-kube-api-access-hgt6b\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.157210 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.156939    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-620f2843-openshift-default-dockercfg-rfg5k\""
Oct 30 18:44:42.181780 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.170697    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"]
Oct 30 18:44:42.226590 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.226546    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istiod-ca-cert\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.226820 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.226802    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-podinfo\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.226980 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.226964    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-credential-socket\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.227102 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.227087    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-envoy\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.227231 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.227218    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-workload-certs\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.227371 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.227357    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgt6b\" (UniqueName: \"kubernetes.io/projected/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-kube-api-access-hgt6b\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.227491 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.227479    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-token\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.227621 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.227606    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-workload-socket\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.227721 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.227710    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-data\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.229765 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.229744    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-data\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.230170 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.230151    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-workload-certs\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.230835 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.230806    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-credential-socket\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.231010 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.230989    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-workload-socket\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.231387 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.231363    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istiod-ca-cert\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.233783 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.233756    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-podinfo\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.234010 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.233989    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-envoy\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.241401 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.241366    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:42.253017 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.247021    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr" event={"ID":"89fdf7cd-72bb-4e56-a826-2df992558d41","Type":"ContainerStarted","Data":"3b84a8a657e33e0d3c1a2fb086581e8a75887e96520293e04585b95368aae126"}
Oct 30 18:44:42.291539 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.291510    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-token\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.291684 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.291510    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgt6b\" (UniqueName: \"kubernetes.io/projected/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-kube-api-access-hgt6b\") pod \"e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") " pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.407921 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.407835    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:42.503439 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.503393    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:42.761512 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.761423    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:42.843758 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.843733    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"]
Oct 30 18:44:42.845115 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:44:42.845083    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf11b4ec3_57bb_46a5_8ea4_79183df95ee2.slice/crio-4da7920c82cb7b2503f22e96d3e6d419fdce73d8a460ef7955047448a664251c WatchSource:0}: Error finding container 4da7920c82cb7b2503f22e96d3e6d419fdce73d8a460ef7955047448a664251c: Status 404 returned error can't find the container with id 4da7920c82cb7b2503f22e96d3e6d419fdce73d8a460ef7955047448a664251c
Oct 30 18:44:42.847669 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.847628    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:44:42.847743 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.847717    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:44:42.847782 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:42.847767    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:44:43.137573 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:43.137526    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:43.253347 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:43.253314    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh" event={"ID":"f11b4ec3-57bb-46a5-8ea4-79183df95ee2","Type":"ContainerStarted","Data":"fc26e0d0be803ba8ecd28ff89e1e8c211650790a9db4a547e34f537a04660658"}
Oct 30 18:44:43.253567 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:43.253370    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh" event={"ID":"f11b4ec3-57bb-46a5-8ea4-79183df95ee2","Type":"ContainerStarted","Data":"4da7920c82cb7b2503f22e96d3e6d419fdce73d8a460ef7955047448a664251c"}
Oct 30 18:44:43.254987 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:43.254943    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr" event={"ID":"89fdf7cd-72bb-4e56-a826-2df992558d41","Type":"ContainerStarted","Data":"031928bab90d546696b001f8c2305aafa5956854e0cd7fd102c46904e74a8c23"}
Oct 30 18:44:43.255117 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:43.255105    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr"
Oct 30 18:44:43.380902 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:43.380841    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh" podStartSLOduration=1.3808246450000001 podStartE2EDuration="1.380824645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:43.380172909 +0000 UTC m=+604.091921607" watchObservedRunningTime="2026-10-30 18:44:43.380824645 +0000 UTC m=+604.092573340"
Oct 30 18:44:43.391176 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:43.391100    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:43.408943 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:43.408920    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:43.413884 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:43.413861    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:43.628447 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:43.628399    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr" podStartSLOduration=2.628382906 podStartE2EDuration="2.628382906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:43.505394946 +0000 UTC m=+604.217143644" watchObservedRunningTime="2026-10-30 18:44:43.628382906 +0000 UTC m=+604.340131602"
Oct 30 18:44:43.934171 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:43.934137    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"]
Oct 30 18:44:43.938065 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:43.938043    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"
Oct 30 18:44:43.944503 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:43.944474    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-maas-api-tls\") pod \"maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf\" (UID: \"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"
Oct 30 18:44:43.944641 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:43.944528    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn9d5\" (UniqueName: \"kubernetes.io/projected/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-kube-api-access-sn9d5\") pod \"maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf\" (UID: \"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"
Oct 30 18:44:43.944709 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:43.944633    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-openshift-service-ca\") pod \"maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf\" (UID: \"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"
Oct 30 18:44:43.944773 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:43.944748    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-metrics-certs\") pod \"maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf\" (UID: \"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"
Oct 30 18:44:43.999131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:43.999092    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"]
Oct 30 18:44:44.045175 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:44.045147    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-metrics-certs\") pod \"maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf\" (UID: \"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"
Oct 30 18:44:44.045319 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:44.045184    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-maas-api-tls\") pod \"maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf\" (UID: \"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"
Oct 30 18:44:44.045319 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:44.045218    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn9d5\" (UniqueName: \"kubernetes.io/projected/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-kube-api-access-sn9d5\") pod \"maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf\" (UID: \"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"
Oct 30 18:44:44.045319 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:44.045278    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-openshift-service-ca\") pod \"maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf\" (UID: \"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"
Oct 30 18:44:44.045941 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:44.045881    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-openshift-service-ca\") pod \"maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf\" (UID: \"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"
Oct 30 18:44:44.047714 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:44.047694    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-metrics-certs\") pod \"maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf\" (UID: \"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"
Oct 30 18:44:44.066813 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:44.066791    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-16a49490-serving-cert\""
Oct 30 18:44:44.075899 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:44:44.075868    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-serving-cert: secret "maas-api-e2e-migrate-16a49490-serving-cert" not found
Oct 30 18:44:44.076060 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:44:44.075971    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-maas-api-tls podName:9f2cf7fe-7d2e-458d-a40f-f8c47be7978d nodeName:}" failed. No retries permitted until 2026-10-30 18:44:44.575933844 +0000 UTC m=+605.287682533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-maas-api-tls") pod "maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf" (UID: "9f2cf7fe-7d2e-458d-a40f-f8c47be7978d") : secret "maas-api-e2e-migrate-16a49490-serving-cert" not found
Oct 30 18:44:44.107504 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:44.107470    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn9d5\" (UniqueName: \"kubernetes.io/projected/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-kube-api-access-sn9d5\") pod \"maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf\" (UID: \"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"
Oct 30 18:44:44.193504 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:44.192856    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:44.261159 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:44.261122    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:44.261849 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:44.261832    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:44:44.401511 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:44.401480    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:44.609556 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:44.609516    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:44.650677 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:44.650637    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-maas-api-tls\") pod \"maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf\" (UID: \"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"
Oct 30 18:44:44.653209 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:44.653187    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-maas-api-tls\") pod \"maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf\" (UID: \"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"
Oct 30 18:44:44.850928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:44.850898    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"
Oct 30 18:44:45.228340 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:45.228315    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"]
Oct 30 18:44:45.230810 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:44:45.230780    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f2cf7fe_7d2e_458d_a40f_f8c47be7978d.slice/crio-e5f521019bb941c2ab72285d8ec14531abaf54fb5f91ce03b6e08f219c8a8df2 WatchSource:0}: Error finding container e5f521019bb941c2ab72285d8ec14531abaf54fb5f91ce03b6e08f219c8a8df2: Status 404 returned error can't find the container with id e5f521019bb941c2ab72285d8ec14531abaf54fb5f91ce03b6e08f219c8a8df2
Oct 30 18:44:45.266428 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:45.266397    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf" event={"ID":"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d","Type":"ContainerStarted","Data":"e5f521019bb941c2ab72285d8ec14531abaf54fb5f91ce03b6e08f219c8a8df2"}
Oct 30 18:44:45.924462 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:45.924425    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:46.108674 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:46.108634    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:46.274570 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:46.274028    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf" event={"ID":"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d","Type":"ContainerStarted","Data":"8b6d9ca9834ebf269f287418f6793a09f0471fa630db98dff025fd9271fd7d1f"}
Oct 30 18:44:47.278918 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:47.278890    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"
Oct 30 18:44:48.749408 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:48.749351    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf" podStartSLOduration=4.800857715 podStartE2EDuration="5.749332832s" totalImagesPullingTime="948.475117ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:46.408015416 +0000 UTC m=+607.119764113" watchObservedRunningTime="2026-10-30 18:44:48.749332832 +0000 UTC m=+609.461081532"
Oct 30 18:44:48.750479 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:48.750459    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"]
Oct 30 18:44:48.754871 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:48.754849    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"
Oct 30 18:44:48.857302 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:48.857265    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-def78c01-serving-cert\""
Oct 30 18:44:48.859241 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:48.859217    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"]
Oct 30 18:44:48.917326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:48.917297    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-maas-api-tls\") pod \"maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb\" (UID: \"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"
Oct 30 18:44:48.917505 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:48.917358    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chbpr\" (UniqueName: \"kubernetes.io/projected/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-kube-api-access-chbpr\") pod \"maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb\" (UID: \"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"
Oct 30 18:44:48.917578 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:48.917536    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-metrics-certs\") pod \"maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb\" (UID: \"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"
Oct 30 18:44:48.917629 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:48.917586    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-openshift-service-ca\") pod \"maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb\" (UID: \"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"
Oct 30 18:44:49.018673 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:49.018577    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-maas-api-tls\") pod \"maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb\" (UID: \"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"
Oct 30 18:44:49.018845 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:49.018677    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chbpr\" (UniqueName: \"kubernetes.io/projected/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-kube-api-access-chbpr\") pod \"maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb\" (UID: \"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"
Oct 30 18:44:49.018845 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:49.018820    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-metrics-certs\") pod \"maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb\" (UID: \"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"
Oct 30 18:44:49.018994 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:49.018866    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-openshift-service-ca\") pod \"maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb\" (UID: \"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"
Oct 30 18:44:49.019427 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:44:49.019404    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-serving-cert: secret "maas-api-e2e-ait-def78c01-serving-cert" not found
Oct 30 18:44:49.019584 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:44:49.019473    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-maas-api-tls podName:a59b4a85-ae70-41f4-92a1-2c2cc9caeee7 nodeName:}" failed. No retries permitted until 2026-10-30 18:44:49.519451339 +0000 UTC m=+610.231200033 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-maas-api-tls") pod "maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb" (UID: "a59b4a85-ae70-41f4-92a1-2c2cc9caeee7") : secret "maas-api-e2e-ait-def78c01-serving-cert" not found
Oct 30 18:44:49.019700 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:49.019668    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-openshift-service-ca\") pod \"maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb\" (UID: \"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"
Oct 30 18:44:49.023772 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:49.023742    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-metrics-certs\") pod \"maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb\" (UID: \"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"
Oct 30 18:44:49.080695 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:49.080651    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chbpr\" (UniqueName: \"kubernetes.io/projected/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-kube-api-access-chbpr\") pod \"maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb\" (UID: \"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"
Oct 30 18:44:49.523294 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:49.523259    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-maas-api-tls\") pod \"maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb\" (UID: \"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"
Oct 30 18:44:49.525713 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:49.525691    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-maas-api-tls\") pod \"maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb\" (UID: \"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"
Oct 30 18:44:49.581021 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:49.580981    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:49.669241 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:49.669212    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"
Oct 30 18:44:49.794519 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:49.794443    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:50.046170 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:50.046100    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"]
Oct 30 18:44:50.050384 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:44:50.050357    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda59b4a85_ae70_41f4_92a1_2c2cc9caeee7.slice/crio-d873fa52dfb5f98102deccd5cf6418fc4090e675a049ce275452054054d0a568 WatchSource:0}: Error finding container d873fa52dfb5f98102deccd5cf6418fc4090e675a049ce275452054054d0a568: Status 404 returned error can't find the container with id d873fa52dfb5f98102deccd5cf6418fc4090e675a049ce275452054054d0a568
Oct 30 18:44:50.300726 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:50.300687    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb" event={"ID":"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7","Type":"ContainerStarted","Data":"d873fa52dfb5f98102deccd5cf6418fc4090e675a049ce275452054054d0a568"}
Oct 30 18:44:51.051821 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:51.051774    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:51.243057 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:51.242925    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:51.316594 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:51.316556    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb" event={"ID":"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7","Type":"ContainerStarted","Data":"1f54e34738343b505fff2090e5fad58e11e1eef0d27663de26f5b17a94220a86"}
Oct 30 18:44:51.316793 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:51.316721    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"
Oct 30 18:44:51.473926 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:51.473852    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb" podStartSLOduration=2.6522792710000003 podStartE2EDuration="3.473832508s" totalImagesPullingTime="821.553237ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:51.470373813 +0000 UTC m=+612.182122516" watchObservedRunningTime="2026-10-30 18:44:51.473832508 +0000 UTC m=+612.185581208"
Oct 30 18:44:53.289471 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:53.289442    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"
Oct 30 18:44:54.043487 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:54.043456    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"]
Oct 30 18:44:54.047281 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:54.047265    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"
Oct 30 18:44:54.069832 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:54.069798    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-openshift-service-ca\") pod \"maas-api-e2e-ait-59144a1b-5467f99466-2hq4v\" (UID: \"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"
Oct 30 18:44:54.069974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:54.069834    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-maas-api-tls\") pod \"maas-api-e2e-ait-59144a1b-5467f99466-2hq4v\" (UID: \"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"
Oct 30 18:44:54.069974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:54.069864    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb845\" (UniqueName: \"kubernetes.io/projected/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-kube-api-access-gb845\") pod \"maas-api-e2e-ait-59144a1b-5467f99466-2hq4v\" (UID: \"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"
Oct 30 18:44:54.070088 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:54.070072    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-metrics-certs\") pod \"maas-api-e2e-ait-59144a1b-5467f99466-2hq4v\" (UID: \"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"
Oct 30 18:44:54.099403 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:54.099379    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-59144a1b-serving-cert\""
Oct 30 18:44:54.152329 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:54.152293    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"]
Oct 30 18:44:54.171219 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:54.171188    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-openshift-service-ca\") pod \"maas-api-e2e-ait-59144a1b-5467f99466-2hq4v\" (UID: \"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"
Oct 30 18:44:54.171337 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:54.171226    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-maas-api-tls\") pod \"maas-api-e2e-ait-59144a1b-5467f99466-2hq4v\" (UID: \"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"
Oct 30 18:44:54.171337 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:54.171246    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb845\" (UniqueName: \"kubernetes.io/projected/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-kube-api-access-gb845\") pod \"maas-api-e2e-ait-59144a1b-5467f99466-2hq4v\" (UID: \"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"
Oct 30 18:44:54.171337 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:54.171318    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-metrics-certs\") pod \"maas-api-e2e-ait-59144a1b-5467f99466-2hq4v\" (UID: \"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"
Oct 30 18:44:54.171517 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:44:54.171350    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-serving-cert: secret "maas-api-e2e-ait-59144a1b-serving-cert" not found
Oct 30 18:44:54.171517 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:44:54.171402    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-maas-api-tls podName:e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c nodeName:}" failed. No retries permitted until 2026-10-30 18:44:54.671388174 +0000 UTC m=+615.383136853 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-maas-api-tls") pod "maas-api-e2e-ait-59144a1b-5467f99466-2hq4v" (UID: "e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c") : secret "maas-api-e2e-ait-59144a1b-serving-cert" not found
Oct 30 18:44:54.171929 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:54.171908    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-openshift-service-ca\") pod \"maas-api-e2e-ait-59144a1b-5467f99466-2hq4v\" (UID: \"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"
Oct 30 18:44:54.173901 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:54.173881    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-metrics-certs\") pod \"maas-api-e2e-ait-59144a1b-5467f99466-2hq4v\" (UID: \"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"
Oct 30 18:44:54.233193 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:54.233166    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb845\" (UniqueName: \"kubernetes.io/projected/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-kube-api-access-gb845\") pod \"maas-api-e2e-ait-59144a1b-5467f99466-2hq4v\" (UID: \"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"
Oct 30 18:44:54.264381 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:54.264359    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr"
Oct 30 18:44:54.286132 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:54.286103    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:54.522911 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:54.522873    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:54.676170 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:54.676132    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-maas-api-tls\") pod \"maas-api-e2e-ait-59144a1b-5467f99466-2hq4v\" (UID: \"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"
Oct 30 18:44:54.678715 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:54.678694    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-maas-api-tls\") pod \"maas-api-e2e-ait-59144a1b-5467f99466-2hq4v\" (UID: \"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"
Oct 30 18:44:54.701079 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:54.701046    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:54.960912 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:54.960881    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"
Oct 30 18:44:55.387476 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:55.387450    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"]
Oct 30 18:44:55.389526 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:44:55.389501    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0c5aedd_56b7_48e0_860b_8ab8fc1fb40c.slice/crio-6f3bf728488664e74b5df4ca8368ee95023f6f12d905b71d132377cc729d324d WatchSource:0}: Error finding container 6f3bf728488664e74b5df4ca8368ee95023f6f12d905b71d132377cc729d324d: Status 404 returned error can't find the container with id 6f3bf728488664e74b5df4ca8368ee95023f6f12d905b71d132377cc729d324d
Oct 30 18:44:56.342405 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:56.342377    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v" event={"ID":"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c","Type":"ContainerStarted","Data":"6f3bf728488664e74b5df4ca8368ee95023f6f12d905b71d132377cc729d324d"}
Oct 30 18:44:57.327145 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:57.327115    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"
Oct 30 18:44:57.365914 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:57.365869    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v" event={"ID":"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c","Type":"ContainerStarted","Data":"bef73c4d6941850742e5a57d16087b98c014eb9993c1ca92505ecf342345cac2"}
Oct 30 18:44:57.366424 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:57.366019    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"
Oct 30 18:44:57.565885 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:57.565837    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v" podStartSLOduration=3.654670851 podStartE2EDuration="4.565820205s" totalImagesPullingTime="911.149354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:57.56406252 +0000 UTC m=+618.275811220" watchObservedRunningTime="2026-10-30 18:44:57.565820205 +0000 UTC m=+618.277569118"
Oct 30 18:44:58.451398 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:58.451362    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:58.687356 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:58.687320    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:59.247396 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:59.247363    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"]
Oct 30 18:44:59.251396 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:59.251380    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"
Oct 30 18:44:59.303360 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:59.303331    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-529e04e2-serving-cert\""
Oct 30 18:44:59.327173 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:59.327144    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d885b1ae-228c-4843-86e3-99d444486027-maas-api-tls\") pod \"maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42\" (UID: \"d885b1ae-228c-4843-86e3-99d444486027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"
Oct 30 18:44:59.327316 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:59.327184    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d885b1ae-228c-4843-86e3-99d444486027-metrics-certs\") pod \"maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42\" (UID: \"d885b1ae-228c-4843-86e3-99d444486027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"
Oct 30 18:44:59.327316 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:59.327286    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d885b1ae-228c-4843-86e3-99d444486027-openshift-service-ca\") pod \"maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42\" (UID: \"d885b1ae-228c-4843-86e3-99d444486027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"
Oct 30 18:44:59.327388 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:59.327332    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwdbk\" (UniqueName: \"kubernetes.io/projected/d885b1ae-228c-4843-86e3-99d444486027-kube-api-access-xwdbk\") pod \"maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42\" (UID: \"d885b1ae-228c-4843-86e3-99d444486027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"
Oct 30 18:44:59.354746 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:59.354717    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"]
Oct 30 18:44:59.428085 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:59.428053    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d885b1ae-228c-4843-86e3-99d444486027-maas-api-tls\") pod \"maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42\" (UID: \"d885b1ae-228c-4843-86e3-99d444486027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"
Oct 30 18:44:59.428085 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:59.428086    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d885b1ae-228c-4843-86e3-99d444486027-metrics-certs\") pod \"maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42\" (UID: \"d885b1ae-228c-4843-86e3-99d444486027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"
Oct 30 18:44:59.428297 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:59.428128    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d885b1ae-228c-4843-86e3-99d444486027-openshift-service-ca\") pod \"maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42\" (UID: \"d885b1ae-228c-4843-86e3-99d444486027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"
Oct 30 18:44:59.428297 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:59.428155    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwdbk\" (UniqueName: \"kubernetes.io/projected/d885b1ae-228c-4843-86e3-99d444486027-kube-api-access-xwdbk\") pod \"maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42\" (UID: \"d885b1ae-228c-4843-86e3-99d444486027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"
Oct 30 18:44:59.428297 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:44:59.428189    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-serving-cert: secret "maas-api-e2e-ait-529e04e2-serving-cert" not found
Oct 30 18:44:59.428297 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:44:59.428249    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d885b1ae-228c-4843-86e3-99d444486027-maas-api-tls podName:d885b1ae-228c-4843-86e3-99d444486027 nodeName:}" failed. No retries permitted until 2026-10-30 18:44:59.928233359 +0000 UTC m=+620.639982049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d885b1ae-228c-4843-86e3-99d444486027-maas-api-tls") pod "maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42" (UID: "d885b1ae-228c-4843-86e3-99d444486027") : secret "maas-api-e2e-ait-529e04e2-serving-cert" not found
Oct 30 18:44:59.428896 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:59.428876    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d885b1ae-228c-4843-86e3-99d444486027-openshift-service-ca\") pod \"maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42\" (UID: \"d885b1ae-228c-4843-86e3-99d444486027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"
Oct 30 18:44:59.430814 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:59.430783    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d885b1ae-228c-4843-86e3-99d444486027-metrics-certs\") pod \"maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42\" (UID: \"d885b1ae-228c-4843-86e3-99d444486027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"
Oct 30 18:44:59.494175 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:59.494140    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwdbk\" (UniqueName: \"kubernetes.io/projected/d885b1ae-228c-4843-86e3-99d444486027-kube-api-access-xwdbk\") pod \"maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42\" (UID: \"d885b1ae-228c-4843-86e3-99d444486027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"
Oct 30 18:44:59.512076 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:59.511992    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:59.703054 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:59.703016    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:59.884599 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:59.884564    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:59.933437 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:59.933407    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d885b1ae-228c-4843-86e3-99d444486027-maas-api-tls\") pod \"maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42\" (UID: \"d885b1ae-228c-4843-86e3-99d444486027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"
Oct 30 18:44:59.935881 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:44:59.935860    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d885b1ae-228c-4843-86e3-99d444486027-maas-api-tls\") pod \"maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42\" (UID: \"d885b1ae-228c-4843-86e3-99d444486027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"
Oct 30 18:45:00.163837 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:00.163741    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"
Oct 30 18:45:00.540385 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:00.540360    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"]
Oct 30 18:45:00.541870 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:45:00.541845    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd885b1ae_228c_4843_86e3_99d444486027.slice/crio-a86178aad798a15a5bf429d38426a8829090b01c51ee2035ec53feb3bb5d65dd WatchSource:0}: Error finding container a86178aad798a15a5bf429d38426a8829090b01c51ee2035ec53feb3bb5d65dd: Status 404 returned error can't find the container with id a86178aad798a15a5bf429d38426a8829090b01c51ee2035ec53feb3bb5d65dd
Oct 30 18:45:01.395224 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:01.395198    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42" event={"ID":"d885b1ae-228c-4843-86e3-99d444486027","Type":"ContainerStarted","Data":"a86178aad798a15a5bf429d38426a8829090b01c51ee2035ec53feb3bb5d65dd"}
Oct 30 18:45:02.402157 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:02.402124    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42" event={"ID":"d885b1ae-228c-4843-86e3-99d444486027","Type":"ContainerStarted","Data":"beac9dc55b310ba342766808de57dcc278b26cee9eb495e2a6934382b27c0c24"}
Oct 30 18:45:02.402554 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:02.402301    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"
Oct 30 18:45:02.521411 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:02.521343    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42" podStartSLOduration=2.69052224 podStartE2EDuration="3.521321897s" totalImagesPullingTime="830.799657ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:02.520419445 +0000 UTC m=+623.232168143" watchObservedRunningTime="2026-10-30 18:45:02.521321897 +0000 UTC m=+623.233070596"
Oct 30 18:45:03.376104 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:03.376075    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"
Oct 30 18:45:04.642806 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:04.642764    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:08.202981 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:45:08.202912    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:45:08.413257 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:08.413224    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"
Oct 30 18:45:09.529655 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:09.529606    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:09.711408 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:09.711379    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:09.893267 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:09.893233    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:14.551975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:14.549628    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:16.504203 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:16.504167    5820 generic.go:356] "Generic (PLEG): container finished" podID="3a5376da-e0ec-4364-98ed-1079db8ac8f7" containerID="f77cdfc7ac1681d95860448512b5812dcd8f5b642413a8de751f4c0f9a5340d2" exitCode=0
Oct 30 18:45:16.504637 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:16.504238    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h" event={"ID":"3a5376da-e0ec-4364-98ed-1079db8ac8f7","Type":"ContainerDied","Data":"f77cdfc7ac1681d95860448512b5812dcd8f5b642413a8de751f4c0f9a5340d2"}
Oct 30 18:45:16.504637 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:16.504632    5820 scope.go:122] "RemoveContainer" containerID="f77cdfc7ac1681d95860448512b5812dcd8f5b642413a8de751f4c0f9a5340d2"
Oct 30 18:45:16.505926 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:16.505896    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:45:16.506066 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:16.505988    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:45:16.506066 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:16.506024    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:45:17.512871 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:17.512797    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h" event={"ID":"3a5376da-e0ec-4364-98ed-1079db8ac8f7","Type":"ContainerStarted","Data":"c2b4485fcc9ec2a3ec572d044f8713db1582517c15b494c5d6e1d286ed509c5d"}
Oct 30 18:45:18.313381 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:18.313346    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:45:18.314417 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:18.314383    5820 patch_prober.go:29] interesting pod/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 30 18:45:18.314548 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:18.314444    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h" podUID="3a5376da-e0ec-4364-98ed-1079db8ac8f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 30 18:45:19.164525 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:19.164489    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:19.313674 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:19.313635    5820 patch_prober.go:29] interesting pod/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 30 18:45:19.313851 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:19.313689    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h" podUID="3a5376da-e0ec-4364-98ed-1079db8ac8f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 30 18:45:20.313591 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:20.313561    5820 patch_prober.go:29] interesting pod/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 30 18:45:20.314115 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:20.314088    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h" podUID="3a5376da-e0ec-4364-98ed-1079db8ac8f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 30 18:45:21.210986 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:21.210927    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:45:21.312855 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:21.312823    5820 patch_prober.go:29] interesting pod/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 30 18:45:21.313048 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:21.312877    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h" podUID="3a5376da-e0ec-4364-98ed-1079db8ac8f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 30 18:45:22.313097 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:22.313065    5820 patch_prober.go:29] interesting pod/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 30 18:45:22.313494 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:22.313123    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h" podUID="3a5376da-e0ec-4364-98ed-1079db8ac8f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 30 18:45:23.318891 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:23.318858    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:45:23.320211 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:23.320187    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:45:24.643678 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:24.638705    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:26.563225 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:26.563180    5820 generic.go:356] "Generic (PLEG): container finished" podID="f11b4ec3-57bb-46a5-8ea4-79183df95ee2" containerID="fc26e0d0be803ba8ecd28ff89e1e8c211650790a9db4a547e34f537a04660658" exitCode=0
Oct 30 18:45:26.563699 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:26.563350    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh" event={"ID":"f11b4ec3-57bb-46a5-8ea4-79183df95ee2","Type":"ContainerDied","Data":"fc26e0d0be803ba8ecd28ff89e1e8c211650790a9db4a547e34f537a04660658"}
Oct 30 18:45:26.563751 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:26.563742    5820 scope.go:122] "RemoveContainer" containerID="fc26e0d0be803ba8ecd28ff89e1e8c211650790a9db4a547e34f537a04660658"
Oct 30 18:45:26.565231 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:26.565198    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:45:26.565332 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:26.565271    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:45:26.565332 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:26.565302    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:45:27.571820 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:27.571782    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh" event={"ID":"f11b4ec3-57bb-46a5-8ea4-79183df95ee2","Type":"ContainerStarted","Data":"6d7aeffdd1b81f6b149b4182807d9dcea48f3511b59e44ab3645a3482e469d79"}
Oct 30 18:45:28.407996 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:28.407932    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:45:28.409271 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:28.409236    5820 patch_prober.go:29] interesting pod/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 18:45:28.409403 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:28.409283    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh" podUID="f11b4ec3-57bb-46a5-8ea4-79183df95ee2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 18:45:29.261976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:29.261071    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:45:29.408522 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:29.408487    5820 patch_prober.go:29] interesting pod/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 18:45:29.408705 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:29.408548    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh" podUID="f11b4ec3-57bb-46a5-8ea4-79183df95ee2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 18:45:30.067296 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:30.067259    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:30.408369 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:30.408284    5820 patch_prober.go:29] interesting pod/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 18:45:30.408369 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:30.408353    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh" podUID="f11b4ec3-57bb-46a5-8ea4-79183df95ee2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 18:45:31.409070 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:31.409033    5820 patch_prober.go:29] interesting pod/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 18:45:31.409480 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:31.409101    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh" podUID="f11b4ec3-57bb-46a5-8ea4-79183df95ee2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 18:45:32.408886 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:32.408849    5820 patch_prober.go:29] interesting pod/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 18:45:32.409093 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:32.408908    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh" podUID="f11b4ec3-57bb-46a5-8ea4-79183df95ee2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 18:45:33.412198 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:33.412166    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:45:33.413251 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:33.413230    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:45:35.299910 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:35.299487    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:35.836537 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:35.836493    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:38.202835 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:45:38.202800    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:45:38.373913 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:38.373874    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:38.548055 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:38.548019    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:40.560967 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:40.560911    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:40.981710 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:40.981629    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:41.169495 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:41.169465    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:44.890421 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:44.890387    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"]
Oct 30 18:45:44.891035 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:44.890576    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf" podUID="9f2cf7fe-7d2e-458d-a40f-f8c47be7978d" containerName="maas-api" containerID="cri-o://8b6d9ca9834ebf269f287418f6793a09f0471fa630db98dff025fd9271fd7d1f" gracePeriod=30
Oct 30 18:45:45.025065 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.025033    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:45.190471 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.190441    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"
Oct 30 18:45:45.222889 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.222840    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-metrics-certs\") pod \"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d\" (UID: \"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d\") "
Oct 30 18:45:45.224136 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.223242    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-maas-api-tls\") pod \"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d\" (UID: \"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d\") "
Oct 30 18:45:45.224136 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.223314    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-openshift-service-ca\") pod \"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d\" (UID: \"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d\") "
Oct 30 18:45:45.224136 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.223448    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-kube-api-access-sn9d5\" (UniqueName: \"kubernetes.io/projected/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-kube-api-access-sn9d5\") pod \"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d\" (UID: \"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d\") "
Oct 30 18:45:45.224136 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.224070    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-openshift-service-ca" pod "9f2cf7fe-7d2e-458d-a40f-f8c47be7978d" (UID: "9f2cf7fe-7d2e-458d-a40f-f8c47be7978d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:45:45.226740 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.226714    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-maas-api-tls" pod "9f2cf7fe-7d2e-458d-a40f-f8c47be7978d" (UID: "9f2cf7fe-7d2e-458d-a40f-f8c47be7978d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:45:45.226850 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.226791    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-metrics-certs" pod "9f2cf7fe-7d2e-458d-a40f-f8c47be7978d" (UID: "9f2cf7fe-7d2e-458d-a40f-f8c47be7978d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:45:45.226920 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.226897    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-kube-api-access-sn9d5" pod "9f2cf7fe-7d2e-458d-a40f-f8c47be7978d" (UID: "9f2cf7fe-7d2e-458d-a40f-f8c47be7978d"). InnerVolumeSpecName "kube-api-access-sn9d5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:45.325543 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.325510    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:45.325543 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.325539    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:45.325786 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.325555    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sn9d5\" (UniqueName: \"kubernetes.io/projected/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-kube-api-access-sn9d5\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:45.325786 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.325565    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:45.692322 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.692284    5820 generic.go:356] "Generic (PLEG): container finished" podID="9f2cf7fe-7d2e-458d-a40f-f8c47be7978d" containerID="8b6d9ca9834ebf269f287418f6793a09f0471fa630db98dff025fd9271fd7d1f" exitCode=0
Oct 30 18:45:45.692486 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.692333    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"
Oct 30 18:45:45.692486 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.692335    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf" event={"ID":"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d","Type":"ContainerDied","Data":"8b6d9ca9834ebf269f287418f6793a09f0471fa630db98dff025fd9271fd7d1f"}
Oct 30 18:45:45.692566 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.692483    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf" event={"ID":"9f2cf7fe-7d2e-458d-a40f-f8c47be7978d","Type":"ContainerDied","Data":"e5f521019bb941c2ab72285d8ec14531abaf54fb5f91ce03b6e08f219c8a8df2"}
Oct 30 18:45:45.692566 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.692525    5820 scope.go:122] "RemoveContainer" containerID="8b6d9ca9834ebf269f287418f6793a09f0471fa630db98dff025fd9271fd7d1f"
Oct 30 18:45:45.701769 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.701747    5820 scope.go:122] "RemoveContainer" containerID="8b6d9ca9834ebf269f287418f6793a09f0471fa630db98dff025fd9271fd7d1f"
Oct 30 18:45:45.702087 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:45:45.702064    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b6d9ca9834ebf269f287418f6793a09f0471fa630db98dff025fd9271fd7d1f\": container with ID starting with 8b6d9ca9834ebf269f287418f6793a09f0471fa630db98dff025fd9271fd7d1f not found: ID does not exist" containerID="8b6d9ca9834ebf269f287418f6793a09f0471fa630db98dff025fd9271fd7d1f"
Oct 30 18:45:45.702153 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.702099    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b6d9ca9834ebf269f287418f6793a09f0471fa630db98dff025fd9271fd7d1f"} err="failed to get container status \"8b6d9ca9834ebf269f287418f6793a09f0471fa630db98dff025fd9271fd7d1f\": rpc error: code = NotFound desc = could not find container \"8b6d9ca9834ebf269f287418f6793a09f0471fa630db98dff025fd9271fd7d1f\": container with ID starting with 8b6d9ca9834ebf269f287418f6793a09f0471fa630db98dff025fd9271fd7d1f not found: ID does not exist"
Oct 30 18:45:45.856818 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.856786    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"]
Oct 30 18:45:45.906435 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:45.906404    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-16a49490-76d6fcb588-l4hsf"]
Oct 30 18:45:46.029842 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.029763    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"]
Oct 30 18:45:46.030031 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.030005    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb" podUID="a59b4a85-ae70-41f4-92a1-2c2cc9caeee7" containerName="maas-api" containerID="cri-o://1f54e34738343b505fff2090e5fad58e11e1eef0d27663de26f5b17a94220a86" gracePeriod=30
Oct 30 18:45:46.170904 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.170845    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:46.312603 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.312573    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"
Oct 30 18:45:46.335936 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.335906    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-openshift-service-ca\") pod \"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7\" (UID: \"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7\") "
Oct 30 18:45:46.336143 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.335968    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-kube-api-access-chbpr\" (UniqueName: \"kubernetes.io/projected/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-kube-api-access-chbpr\") pod \"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7\" (UID: \"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7\") "
Oct 30 18:45:46.336143 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.336000    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-metrics-certs\") pod \"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7\" (UID: \"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7\") "
Oct 30 18:45:46.336143 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.336115    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-maas-api-tls\") pod \"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7\" (UID: \"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7\") "
Oct 30 18:45:46.336583 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.336380    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-openshift-service-ca" pod "a59b4a85-ae70-41f4-92a1-2c2cc9caeee7" (UID: "a59b4a85-ae70-41f4-92a1-2c2cc9caeee7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:45:46.336583 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.336556    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:46.338451 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.338421    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-maas-api-tls" pod "a59b4a85-ae70-41f4-92a1-2c2cc9caeee7" (UID: "a59b4a85-ae70-41f4-92a1-2c2cc9caeee7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:45:46.338567 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.338540    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-kube-api-access-chbpr" pod "a59b4a85-ae70-41f4-92a1-2c2cc9caeee7" (UID: "a59b4a85-ae70-41f4-92a1-2c2cc9caeee7"). InnerVolumeSpecName "kube-api-access-chbpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:46.338610 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.338554    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-metrics-certs" pod "a59b4a85-ae70-41f4-92a1-2c2cc9caeee7" (UID: "a59b4a85-ae70-41f4-92a1-2c2cc9caeee7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:45:46.442971 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.442008    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chbpr\" (UniqueName: \"kubernetes.io/projected/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-kube-api-access-chbpr\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:46.442971 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.442047    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:46.442971 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.442062    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:46.705662 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.705612    5820 generic.go:356] "Generic (PLEG): container finished" podID="a59b4a85-ae70-41f4-92a1-2c2cc9caeee7" containerID="1f54e34738343b505fff2090e5fad58e11e1eef0d27663de26f5b17a94220a86" exitCode=0
Oct 30 18:45:46.705860 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.705680    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"
Oct 30 18:45:46.705860 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.705701    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb" event={"ID":"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7","Type":"ContainerDied","Data":"1f54e34738343b505fff2090e5fad58e11e1eef0d27663de26f5b17a94220a86"}
Oct 30 18:45:46.705860 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.705805    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb" event={"ID":"a59b4a85-ae70-41f4-92a1-2c2cc9caeee7","Type":"ContainerDied","Data":"d873fa52dfb5f98102deccd5cf6418fc4090e675a049ce275452054054d0a568"}
Oct 30 18:45:46.705860 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.705833    5820 scope.go:122] "RemoveContainer" containerID="1f54e34738343b505fff2090e5fad58e11e1eef0d27663de26f5b17a94220a86"
Oct 30 18:45:46.717101 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.717081    5820 scope.go:122] "RemoveContainer" containerID="1f54e34738343b505fff2090e5fad58e11e1eef0d27663de26f5b17a94220a86"
Oct 30 18:45:46.717417 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:45:46.717396    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f54e34738343b505fff2090e5fad58e11e1eef0d27663de26f5b17a94220a86\": container with ID starting with 1f54e34738343b505fff2090e5fad58e11e1eef0d27663de26f5b17a94220a86 not found: ID does not exist" containerID="1f54e34738343b505fff2090e5fad58e11e1eef0d27663de26f5b17a94220a86"
Oct 30 18:45:46.717487 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.717426    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f54e34738343b505fff2090e5fad58e11e1eef0d27663de26f5b17a94220a86"} err="failed to get container status \"1f54e34738343b505fff2090e5fad58e11e1eef0d27663de26f5b17a94220a86\": rpc error: code = NotFound desc = could not find container \"1f54e34738343b505fff2090e5fad58e11e1eef0d27663de26f5b17a94220a86\": container with ID starting with 1f54e34738343b505fff2090e5fad58e11e1eef0d27663de26f5b17a94220a86 not found: ID does not exist"
Oct 30 18:45:46.871617 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.871589    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"]
Oct 30 18:45:46.874347 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.874318    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-def78c01-5c99d858ff-zlzjb"]
Oct 30 18:45:46.905102 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:46.905069    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:47.209996 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.208686    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:47.227777 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.227744    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"]
Oct 30 18:45:47.228033 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.228010    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v" podUID="e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c" containerName="maas-api" containerID="cri-o://bef73c4d6941850742e5a57d16087b98c014eb9993c1ca92505ecf342345cac2" gracePeriod=30
Oct 30 18:45:47.405264 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.405225    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:47.437692 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.437661    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f2cf7fe-7d2e-458d-a40f-f8c47be7978d" path="/var/lib/kubelet/pods/9f2cf7fe-7d2e-458d-a40f-f8c47be7978d/volumes"
Oct 30 18:45:47.438177 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.438161    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a59b4a85-ae70-41f4-92a1-2c2cc9caeee7" path="/var/lib/kubelet/pods/a59b4a85-ae70-41f4-92a1-2c2cc9caeee7/volumes"
Oct 30 18:45:47.496517 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.496487    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"
Oct 30 18:45:47.501457 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.501429    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"]
Oct 30 18:45:47.501631 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.501610    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66" podUID="358d8765-c33a-4934-a833-2d6fe588780e" containerName="istio-proxy" containerID="cri-o://cc3b8eb6adc7a7eba2e44c66e0d4eb6a97a340541c0eafd725094610f5de3c91" gracePeriod=30
Oct 30 18:45:47.556943 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.556909    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-metrics-certs\") pod \"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c\" (UID: \"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c\") "
Oct 30 18:45:47.557136 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.557019    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-maas-api-tls\") pod \"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c\" (UID: \"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c\") "
Oct 30 18:45:47.557136 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.557110    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-kube-api-access-gb845\" (UniqueName: \"kubernetes.io/projected/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-kube-api-access-gb845\") pod \"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c\" (UID: \"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c\") "
Oct 30 18:45:47.557252 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.557194    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-openshift-service-ca\") pod \"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c\" (UID: \"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c\") "
Oct 30 18:45:47.557649 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.557621    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-openshift-service-ca" pod "e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c" (UID: "e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:45:47.559404 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.559385    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-kube-api-access-gb845" pod "e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c" (UID: "e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c"). InnerVolumeSpecName "kube-api-access-gb845". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:47.559674 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.559646    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-maas-api-tls" pod "e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c" (UID: "e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:45:47.559674 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.559661    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-metrics-certs" pod "e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c" (UID: "e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:45:47.592062 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.592029    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:47.658567 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.658532    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:47.658567 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.658565    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:47.658808 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.658578    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gb845\" (UniqueName: \"kubernetes.io/projected/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-kube-api-access-gb845\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:47.658808 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.658591    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:47.714192 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.714160    5820 generic.go:356] "Generic (PLEG): container finished" podID="e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c" containerID="bef73c4d6941850742e5a57d16087b98c014eb9993c1ca92505ecf342345cac2" exitCode=0
Oct 30 18:45:47.714364 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.714203    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v" event={"ID":"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c","Type":"ContainerDied","Data":"bef73c4d6941850742e5a57d16087b98c014eb9993c1ca92505ecf342345cac2"}
Oct 30 18:45:47.714364 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.714223    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"
Oct 30 18:45:47.714364 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.714257    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v" event={"ID":"e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c","Type":"ContainerDied","Data":"6f3bf728488664e74b5df4ca8368ee95023f6f12d905b71d132377cc729d324d"}
Oct 30 18:45:47.714364 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.714282    5820 scope.go:122] "RemoveContainer" containerID="bef73c4d6941850742e5a57d16087b98c014eb9993c1ca92505ecf342345cac2"
Oct 30 18:45:47.724172 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.724150    5820 scope.go:122] "RemoveContainer" containerID="bef73c4d6941850742e5a57d16087b98c014eb9993c1ca92505ecf342345cac2"
Oct 30 18:45:47.724483 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:45:47.724462    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bef73c4d6941850742e5a57d16087b98c014eb9993c1ca92505ecf342345cac2\": container with ID starting with bef73c4d6941850742e5a57d16087b98c014eb9993c1ca92505ecf342345cac2 not found: ID does not exist" containerID="bef73c4d6941850742e5a57d16087b98c014eb9993c1ca92505ecf342345cac2"
Oct 30 18:45:47.724546 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.724494    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bef73c4d6941850742e5a57d16087b98c014eb9993c1ca92505ecf342345cac2"} err="failed to get container status \"bef73c4d6941850742e5a57d16087b98c014eb9993c1ca92505ecf342345cac2\": rpc error: code = NotFound desc = could not find container \"bef73c4d6941850742e5a57d16087b98c014eb9993c1ca92505ecf342345cac2\": container with ID starting with bef73c4d6941850742e5a57d16087b98c014eb9993c1ca92505ecf342345cac2 not found: ID does not exist"
Oct 30 18:45:47.792603 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.785173    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:47.848038 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.848007    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"]
Oct 30 18:45:47.855678 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.855650    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-59144a1b-5467f99466-2hq4v"]
Oct 30 18:45:47.968575 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:47.968537    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:48.146050 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.146013    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:48.321142 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.321105    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:48.539841 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.539801    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:48.724509 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.724475    5820 generic.go:356] "Generic (PLEG): container finished" podID="358d8765-c33a-4934-a833-2d6fe588780e" containerID="cc3b8eb6adc7a7eba2e44c66e0d4eb6a97a340541c0eafd725094610f5de3c91" exitCode=0
Oct 30 18:45:48.724675 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.724520    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66" event={"ID":"358d8765-c33a-4934-a833-2d6fe588780e","Type":"ContainerDied","Data":"cc3b8eb6adc7a7eba2e44c66e0d4eb6a97a340541c0eafd725094610f5de3c91"}
Oct 30 18:45:48.756108 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.756086    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:45:48.871039 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.871003    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/358d8765-c33a-4934-a833-2d6fe588780e-istio-token\" (UniqueName: \"kubernetes.io/projected/358d8765-c33a-4934-a833-2d6fe588780e-istio-token\") pod \"358d8765-c33a-4934-a833-2d6fe588780e\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") "
Oct 30 18:45:48.871247 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.871059    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-credential-socket\") pod \"358d8765-c33a-4934-a833-2d6fe588780e\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") "
Oct 30 18:45:48.871247 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.871153    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-istio-envoy\") pod \"358d8765-c33a-4934-a833-2d6fe588780e\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") "
Oct 30 18:45:48.871247 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.871189    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-istio-data\") pod \"358d8765-c33a-4934-a833-2d6fe588780e\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") "
Oct 30 18:45:48.871247 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.871240    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-workload-socket\") pod \"358d8765-c33a-4934-a833-2d6fe588780e\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") "
Oct 30 18:45:48.871473 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.871266    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/358d8765-c33a-4934-a833-2d6fe588780e-kube-api-access-kncn8\" (UniqueName: \"kubernetes.io/projected/358d8765-c33a-4934-a833-2d6fe588780e-kube-api-access-kncn8\") pod \"358d8765-c33a-4934-a833-2d6fe588780e\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") "
Oct 30 18:45:48.871473 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.871301    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/358d8765-c33a-4934-a833-2d6fe588780e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/358d8765-c33a-4934-a833-2d6fe588780e-istio-podinfo\") pod \"358d8765-c33a-4934-a833-2d6fe588780e\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") "
Oct 30 18:45:48.871473 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.871347    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/358d8765-c33a-4934-a833-2d6fe588780e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/358d8765-c33a-4934-a833-2d6fe588780e-istiod-ca-cert\") pod \"358d8765-c33a-4934-a833-2d6fe588780e\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") "
Oct 30 18:45:48.871473 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.871449    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-workload-certs\") pod \"358d8765-c33a-4934-a833-2d6fe588780e\" (UID: \"358d8765-c33a-4934-a833-2d6fe588780e\") "
Oct 30 18:45:48.871473 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.871455    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-credential-socket" pod "358d8765-c33a-4934-a833-2d6fe588780e" (UID: "358d8765-c33a-4934-a833-2d6fe588780e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:45:48.871737 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.871528    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-workload-socket" pod "358d8765-c33a-4934-a833-2d6fe588780e" (UID: "358d8765-c33a-4934-a833-2d6fe588780e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:45:48.872064 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.871832    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:48.872064 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.871856    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:48.872064 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.871866    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-workload-certs" pod "358d8765-c33a-4934-a833-2d6fe588780e" (UID: "358d8765-c33a-4934-a833-2d6fe588780e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:45:48.872064 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.871982    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/358d8765-c33a-4934-a833-2d6fe588780e-istiod-ca-cert" pod "358d8765-c33a-4934-a833-2d6fe588780e" (UID: "358d8765-c33a-4934-a833-2d6fe588780e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:45:48.872064 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.872075    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-istio-data" pod "358d8765-c33a-4934-a833-2d6fe588780e" (UID: "358d8765-c33a-4934-a833-2d6fe588780e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:45:48.873812 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.873785    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-istio-envoy" pod "358d8765-c33a-4934-a833-2d6fe588780e" (UID: "358d8765-c33a-4934-a833-2d6fe588780e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:45:48.873926 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.873858    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/358d8765-c33a-4934-a833-2d6fe588780e-istio-token" pod "358d8765-c33a-4934-a833-2d6fe588780e" (UID: "358d8765-c33a-4934-a833-2d6fe588780e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:48.874257 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.874238    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/358d8765-c33a-4934-a833-2d6fe588780e-istio-podinfo" pod "358d8765-c33a-4934-a833-2d6fe588780e" (UID: "358d8765-c33a-4934-a833-2d6fe588780e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:45:48.874334 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.874301    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/358d8765-c33a-4934-a833-2d6fe588780e-kube-api-access-kncn8" pod "358d8765-c33a-4934-a833-2d6fe588780e" (UID: "358d8765-c33a-4934-a833-2d6fe588780e"). InnerVolumeSpecName "kube-api-access-kncn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:48.905888 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.905859    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"]
Oct 30 18:45:48.906353 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.906340    5820 cpu_manager.go:414] "containerMap: removing container" podUID="9f2cf7fe-7d2e-458d-a40f-f8c47be7978d" containerName="maas-api"
Oct 30 18:45:48.906413 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.906355    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f2cf7fe-7d2e-458d-a40f-f8c47be7978d" containerName="maas-api"
Oct 30 18:45:48.906413 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.906377    5820 cpu_manager.go:414] "containerMap: removing container" podUID="e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c" containerName="maas-api"
Oct 30 18:45:48.906413 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.906382    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c" containerName="maas-api"
Oct 30 18:45:48.906413 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.906393    5820 cpu_manager.go:414] "containerMap: removing container" podUID="358d8765-c33a-4934-a833-2d6fe588780e" containerName="istio-proxy"
Oct 30 18:45:48.906413 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.906397    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="358d8765-c33a-4934-a833-2d6fe588780e" containerName="istio-proxy"
Oct 30 18:45:48.906413 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.906403    5820 cpu_manager.go:414] "containerMap: removing container" podUID="a59b4a85-ae70-41f4-92a1-2c2cc9caeee7" containerName="maas-api"
Oct 30 18:45:48.906413 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.906408    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a59b4a85-ae70-41f4-92a1-2c2cc9caeee7" containerName="maas-api"
Oct 30 18:45:48.906648 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.906470    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c" containerName="maas-api"
Oct 30 18:45:48.906648 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.906478    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="358d8765-c33a-4934-a833-2d6fe588780e" containerName="istio-proxy"
Oct 30 18:45:48.906648 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.906484    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="a59b4a85-ae70-41f4-92a1-2c2cc9caeee7" containerName="maas-api"
Oct 30 18:45:48.906648 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.906493    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f2cf7fe-7d2e-458d-a40f-f8c47be7978d" containerName="maas-api"
Oct 30 18:45:48.909731 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.909715    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:48.962081 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.962051    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-3b6013-openshift-default-dockercfg-4sf9f\""
Oct 30 18:45:48.975585 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.975545    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-workload-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:48.975767 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.975597    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-credential-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:48.975767 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.975689    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istiod-ca-cert\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:48.975767 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.975715    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-data\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:48.975935 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.975777    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-workload-certs\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:48.975935 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.975822    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-envoy\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:48.975935 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.975882    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-podinfo\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:48.975935 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.975906    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7tz8\" (UniqueName: \"kubernetes.io/projected/6a253f0b-1803-4c70-b5af-aa295e8c47cb-kube-api-access-t7tz8\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:48.976167 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.975935    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-token\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:48.976167 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.976007    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:48.976167 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.976024    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:48.976167 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.976039    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kncn8\" (UniqueName: \"kubernetes.io/projected/358d8765-c33a-4934-a833-2d6fe588780e-kube-api-access-kncn8\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:48.976167 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.976053    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/358d8765-c33a-4934-a833-2d6fe588780e-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:48.976167 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.976066    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/358d8765-c33a-4934-a833-2d6fe588780e-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:48.976167 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.976079    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/358d8765-c33a-4934-a833-2d6fe588780e-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:48.976167 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.976095    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/358d8765-c33a-4934-a833-2d6fe588780e-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:48.994517 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:48.994487    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"]
Oct 30 18:45:49.048344 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.048304    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:49.077284 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.076734    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-data\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:49.077284 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.076815    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-workload-certs\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:49.077284 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.076862    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-envoy\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:49.077284 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.076921    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-podinfo\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:49.077284 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.076964    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7tz8\" (UniqueName: \"kubernetes.io/projected/6a253f0b-1803-4c70-b5af-aa295e8c47cb-kube-api-access-t7tz8\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:49.077284 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.076995    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-token\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:49.077284 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.077046    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-workload-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:49.077284 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.077071    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-credential-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:49.077284 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.077163    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istiod-ca-cert\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:49.077805 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.077783    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istiod-ca-cert\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:49.078390 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.078032    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-data\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:49.078390 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.078286    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-workload-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:49.078390 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.078363    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-workload-certs\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:49.078625 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.078420    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-credential-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:49.080754 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.080732    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-envoy\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:49.081142 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.081120    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-podinfo\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:49.139642 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.139575    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-token\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:49.139764 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.139699    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7tz8\" (UniqueName: \"kubernetes.io/projected/6a253f0b-1803-4c70-b5af-aa295e8c47cb-kube-api-access-t7tz8\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:49.223415 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.223380    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:49.227232 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.227135    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:49.411662 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.411575    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:49.445225 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.445189    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c" path="/var/lib/kubelet/pods/e0c5aedd-56b7-48e0-860b-8ab8fc1fb40c/volumes"
Oct 30 18:45:49.597487 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.597456    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:49.688141 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.688116    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"]
Oct 30 18:45:49.690465 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:45:49.690441    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a253f0b_1803_4c70_b5af_aa295e8c47cb.slice/crio-ff87535ea84dc8ea4c50e4da19783cecd6d4b66af7fae6eff7b490d0b462cc4c WatchSource:0}: Error finding container ff87535ea84dc8ea4c50e4da19783cecd6d4b66af7fae6eff7b490d0b462cc4c: Status 404 returned error can't find the container with id ff87535ea84dc8ea4c50e4da19783cecd6d4b66af7fae6eff7b490d0b462cc4c
Oct 30 18:45:49.693000 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.692944    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:45:49.693073 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.693055    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:45:49.693114 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.693102    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:45:49.730606 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.730576    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"
Oct 30 18:45:49.730713 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.730577    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66" event={"ID":"358d8765-c33a-4934-a833-2d6fe588780e","Type":"ContainerDied","Data":"8c28e028d88bb00f65c7c305a63d05578301d6af12659f3137c57a7bdadbe07d"}
Oct 30 18:45:49.730713 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.730677    5820 scope.go:122] "RemoveContainer" containerID="cc3b8eb6adc7a7eba2e44c66e0d4eb6a97a340541c0eafd725094610f5de3c91"
Oct 30 18:45:49.732201 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.732139    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz" event={"ID":"6a253f0b-1803-4c70-b5af-aa295e8c47cb","Type":"ContainerStarted","Data":"ff87535ea84dc8ea4c50e4da19783cecd6d4b66af7fae6eff7b490d0b462cc4c"}
Oct 30 18:45:49.896104 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.896069    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"]
Oct 30 18:45:49.901231 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.901207    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-16a49490-gw-openshift-default-687cf867c7-fkv66"]
Oct 30 18:45:49.917968 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:49.917912    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:50.094714 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.094678    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:50.478267 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.478184    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"]
Oct 30 18:45:50.482240 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.482213    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.540975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.536229    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-2ade09-openshift-default-dockercfg-8v89k\""
Oct 30 18:45:50.587028 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.586994    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"]
Oct 30 18:45:50.593025 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.592996    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.593165 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.593035    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.593165 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.593087    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.593259 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.593170    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.593322 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.593304    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.593370 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.593331    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.593370 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.593359    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxzlk\" (UniqueName: \"kubernetes.io/projected/4be0753d-602a-4f4a-a02d-5d6baf686df2-kube-api-access-qxzlk\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.593448 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.593437    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4be0753d-602a-4f4a-a02d-5d6baf686df2-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.593480 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.593463    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.606731 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.606704    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:50.693978 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.693930    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.694142 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.694013    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.694142 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.694035    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.694142 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.694056    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxzlk\" (UniqueName: \"kubernetes.io/projected/4be0753d-602a-4f4a-a02d-5d6baf686df2-kube-api-access-qxzlk\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.694142 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.694095    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4be0753d-602a-4f4a-a02d-5d6baf686df2-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.694385 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.694242    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.694385 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.694334    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.694385 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.694366    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.694536 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.694401    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.694536 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.694471    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.694651 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.694556    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.694795 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.694773    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.694795 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.694789    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.695166 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.695140    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4be0753d-602a-4f4a-a02d-5d6baf686df2-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.696712 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.696693    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.696776 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.696732    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.740758 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.740685    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz" event={"ID":"6a253f0b-1803-4c70-b5af-aa295e8c47cb","Type":"ContainerStarted","Data":"1e8c99c5b86a3fde4d5718e0bf838f84c9e76a8dcd5ae60f6cd13e524a2a4e87"}
Oct 30 18:45:50.754881 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.754857    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.755084 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.755066    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxzlk\" (UniqueName: \"kubernetes.io/projected/4be0753d-602a-4f4a-a02d-5d6baf686df2-kube-api-access-qxzlk\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.794999 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.794970    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:50.830788 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.830754    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:50.862694 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.862626    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz" podStartSLOduration=2.8626118 podStartE2EDuration="2.8626118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:50.862303797 +0000 UTC m=+671.574052495" watchObservedRunningTime="2026-10-30 18:45:50.8626118 +0000 UTC m=+671.574360498"
Oct 30 18:45:50.875122 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:50.875092    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr"]
Oct 30 18:45:51.223811 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:51.223771    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:51.225505 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:51.225475    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"]
Oct 30 18:45:51.227748 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:45:51.227719    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4be0753d_602a_4f4a_a02d_5d6baf686df2.slice/crio-eeecf01a8c36e917c8ff8037d0a5ff95c26d42ef89ad13de02e361f554beeaa5 WatchSource:0}: Error finding container eeecf01a8c36e917c8ff8037d0a5ff95c26d42ef89ad13de02e361f554beeaa5: Status 404 returned error can't find the container with id eeecf01a8c36e917c8ff8037d0a5ff95c26d42ef89ad13de02e361f554beeaa5
Oct 30 18:45:51.229562 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:51.229549    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:51.230383 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:51.230347    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:45:51.230473 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:51.230432    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:45:51.230546 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:51.230480    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:45:51.298894 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:51.298847    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:51.450594 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:51.450554    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="358d8765-c33a-4934-a833-2d6fe588780e" path="/var/lib/kubelet/pods/358d8765-c33a-4934-a833-2d6fe588780e/volumes"
Oct 30 18:45:51.472076 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:51.472040    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:51.794977 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:51.794920    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6" event={"ID":"4be0753d-602a-4f4a-a02d-5d6baf686df2","Type":"ContainerStarted","Data":"fb8d8751512c45691d330f0e75dd6a4ab125a2bf91b0f276cfcae2350c47c972"}
Oct 30 18:45:51.795485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:51.795003    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:51.795485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:51.795032    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6" event={"ID":"4be0753d-602a-4f4a-a02d-5d6baf686df2","Type":"ContainerStarted","Data":"eeecf01a8c36e917c8ff8037d0a5ff95c26d42ef89ad13de02e361f554beeaa5"}
Oct 30 18:45:51.795616 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:51.795516    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:51.796124 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:51.796064    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:45:51.915533 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:51.915446    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6" podStartSLOduration=1.91542445 podStartE2EDuration="1.91542445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:51.91433162 +0000 UTC m=+672.626080314" watchObservedRunningTime="2026-10-30 18:45:51.91542445 +0000 UTC m=+672.627173149"
Oct 30 18:45:52.800592 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:52.800565    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:53.758919 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:53.758885    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:53.804031 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:53.803986    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:53.804681 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:53.804662    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:45:53.940722 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:53.940684    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:54.126115 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:54.126084    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:54.608975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:54.608832    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:54.794042 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:54.794013    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:54.972025 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:54.971927    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:55.875854 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:55.875818    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr" podUID="89fdf7cd-72bb-4e56-a826-2df992558d41" containerName="payload-processing" containerID="cri-o://031928bab90d546696b001f8c2305aafa5956854e0cd7fd102c46904e74a8c23" gracePeriod=25
Oct 30 18:45:56.139508 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:56.139449    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr"
Oct 30 18:45:56.274474 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:56.274446    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/89fdf7cd-72bb-4e56-a826-2df992558d41-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89fdf7cd-72bb-4e56-a826-2df992558d41-plugins-config-volume\") pod \"89fdf7cd-72bb-4e56-a826-2df992558d41\" (UID: \"89fdf7cd-72bb-4e56-a826-2df992558d41\") "
Oct 30 18:45:56.274638 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:56.274535    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89fdf7cd-72bb-4e56-a826-2df992558d41-kube-api-access-dpvpt\" (UniqueName: \"kubernetes.io/projected/89fdf7cd-72bb-4e56-a826-2df992558d41-kube-api-access-dpvpt\") pod \"89fdf7cd-72bb-4e56-a826-2df992558d41\" (UID: \"89fdf7cd-72bb-4e56-a826-2df992558d41\") "
Oct 30 18:45:56.274910 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:56.274879    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89fdf7cd-72bb-4e56-a826-2df992558d41-plugins-config-volume" pod "89fdf7cd-72bb-4e56-a826-2df992558d41" (UID: "89fdf7cd-72bb-4e56-a826-2df992558d41"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:45:56.276764 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:56.276745    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89fdf7cd-72bb-4e56-a826-2df992558d41-kube-api-access-dpvpt" pod "89fdf7cd-72bb-4e56-a826-2df992558d41" (UID: "89fdf7cd-72bb-4e56-a826-2df992558d41"). InnerVolumeSpecName "kube-api-access-dpvpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:56.376256 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:56.376221    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpvpt\" (UniqueName: \"kubernetes.io/projected/89fdf7cd-72bb-4e56-a826-2df992558d41-kube-api-access-dpvpt\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:56.376256 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:56.376251    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89fdf7cd-72bb-4e56-a826-2df992558d41-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:56.786512 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:56.786478    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:56.821033 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:56.820996    5820 generic.go:356] "Generic (PLEG): container finished" podID="89fdf7cd-72bb-4e56-a826-2df992558d41" containerID="031928bab90d546696b001f8c2305aafa5956854e0cd7fd102c46904e74a8c23" exitCode=0
Oct 30 18:45:56.821191 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:56.821050    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr"
Oct 30 18:45:56.821191 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:56.821074    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr" event={"ID":"89fdf7cd-72bb-4e56-a826-2df992558d41","Type":"ContainerDied","Data":"031928bab90d546696b001f8c2305aafa5956854e0cd7fd102c46904e74a8c23"}
Oct 30 18:45:56.821191 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:56.821124    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr" event={"ID":"89fdf7cd-72bb-4e56-a826-2df992558d41","Type":"ContainerDied","Data":"3b84a8a657e33e0d3c1a2fb086581e8a75887e96520293e04585b95368aae126"}
Oct 30 18:45:56.821191 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:56.821145    5820 scope.go:122] "RemoveContainer" containerID="031928bab90d546696b001f8c2305aafa5956854e0cd7fd102c46904e74a8c23"
Oct 30 18:45:56.831144 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:56.831121    5820 scope.go:122] "RemoveContainer" containerID="031928bab90d546696b001f8c2305aafa5956854e0cd7fd102c46904e74a8c23"
Oct 30 18:45:56.831417 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:45:56.831398    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"031928bab90d546696b001f8c2305aafa5956854e0cd7fd102c46904e74a8c23\": container with ID starting with 031928bab90d546696b001f8c2305aafa5956854e0cd7fd102c46904e74a8c23 not found: ID does not exist" containerID="031928bab90d546696b001f8c2305aafa5956854e0cd7fd102c46904e74a8c23"
Oct 30 18:45:56.831461 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:56.831427    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"031928bab90d546696b001f8c2305aafa5956854e0cd7fd102c46904e74a8c23"} err="failed to get container status \"031928bab90d546696b001f8c2305aafa5956854e0cd7fd102c46904e74a8c23\": rpc error: code = NotFound desc = could not find container \"031928bab90d546696b001f8c2305aafa5956854e0cd7fd102c46904e74a8c23\": container with ID starting with 031928bab90d546696b001f8c2305aafa5956854e0cd7fd102c46904e74a8c23 not found: ID does not exist"
Oct 30 18:45:56.988981 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:56.988925    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr"]
Oct 30 18:45:56.996761 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:56.996734    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-529e04e2-85c9cc8486-7vkwr"]
Oct 30 18:45:57.062846 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:57.062816    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:57.242179 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:57.242143    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:57.438724 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:57.438633    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:57.463213 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:57.463178    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="89fdf7cd-72bb-4e56-a826-2df992558d41" path="/var/lib/kubelet/pods/89fdf7cd-72bb-4e56-a826-2df992558d41/volumes"
Oct 30 18:45:58.228563 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:58.228518    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:58.855763 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:58.855721    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:59.079934 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:59.079899    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:59.877408 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:45:59.877366    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:01.075396 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:01.075363    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:01.306487 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:01.306452    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:03.165327 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:03.165287    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:03.393746 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:03.393708    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:04.082131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.082099    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"]
Oct 30 18:46:04.082349 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.082323    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42" podUID="d885b1ae-228c-4843-86e3-99d444486027" containerName="maas-api" containerID="cri-o://beac9dc55b310ba342766808de57dcc278b26cee9eb495e2a6934382b27c0c24" gracePeriod=30
Oct 30 18:46:04.218272 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.218239    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:04.368216 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.368192    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"
Oct 30 18:46:04.500934 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.499976    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d885b1ae-228c-4843-86e3-99d444486027-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d885b1ae-228c-4843-86e3-99d444486027-openshift-service-ca\") pod \"d885b1ae-228c-4843-86e3-99d444486027\" (UID: \"d885b1ae-228c-4843-86e3-99d444486027\") "
Oct 30 18:46:04.500934 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.500114    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d885b1ae-228c-4843-86e3-99d444486027-kube-api-access-xwdbk\" (UniqueName: \"kubernetes.io/projected/d885b1ae-228c-4843-86e3-99d444486027-kube-api-access-xwdbk\") pod \"d885b1ae-228c-4843-86e3-99d444486027\" (UID: \"d885b1ae-228c-4843-86e3-99d444486027\") "
Oct 30 18:46:04.500934 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.500170    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d885b1ae-228c-4843-86e3-99d444486027-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d885b1ae-228c-4843-86e3-99d444486027-maas-api-tls\") pod \"d885b1ae-228c-4843-86e3-99d444486027\" (UID: \"d885b1ae-228c-4843-86e3-99d444486027\") "
Oct 30 18:46:04.500934 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.500196    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d885b1ae-228c-4843-86e3-99d444486027-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d885b1ae-228c-4843-86e3-99d444486027-metrics-certs\") pod \"d885b1ae-228c-4843-86e3-99d444486027\" (UID: \"d885b1ae-228c-4843-86e3-99d444486027\") "
Oct 30 18:46:04.503274 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.502861    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d885b1ae-228c-4843-86e3-99d444486027-openshift-service-ca" pod "d885b1ae-228c-4843-86e3-99d444486027" (UID: "d885b1ae-228c-4843-86e3-99d444486027"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:46:04.510521 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.510471    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d885b1ae-228c-4843-86e3-99d444486027-maas-api-tls" pod "d885b1ae-228c-4843-86e3-99d444486027" (UID: "d885b1ae-228c-4843-86e3-99d444486027"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:04.510856 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.510819    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d885b1ae-228c-4843-86e3-99d444486027-metrics-certs" pod "d885b1ae-228c-4843-86e3-99d444486027" (UID: "d885b1ae-228c-4843-86e3-99d444486027"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:04.517614 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.517227    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d885b1ae-228c-4843-86e3-99d444486027-kube-api-access-xwdbk" pod "d885b1ae-228c-4843-86e3-99d444486027" (UID: "d885b1ae-228c-4843-86e3-99d444486027"). InnerVolumeSpecName "kube-api-access-xwdbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:46:04.601215 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.601143    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwdbk\" (UniqueName: \"kubernetes.io/projected/d885b1ae-228c-4843-86e3-99d444486027-kube-api-access-xwdbk\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:04.601215 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.601169    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d885b1ae-228c-4843-86e3-99d444486027-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:04.601215 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.601179    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d885b1ae-228c-4843-86e3-99d444486027-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:04.601215 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.601187    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d885b1ae-228c-4843-86e3-99d444486027-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:04.639918 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.639885    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:04.818719 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.818678    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:04.869748 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.869654    5820 generic.go:356] "Generic (PLEG): container finished" podID="d885b1ae-228c-4843-86e3-99d444486027" containerID="beac9dc55b310ba342766808de57dcc278b26cee9eb495e2a6934382b27c0c24" exitCode=0
Oct 30 18:46:04.869748 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.869706    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"
Oct 30 18:46:04.869748 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.869718    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42" event={"ID":"d885b1ae-228c-4843-86e3-99d444486027","Type":"ContainerDied","Data":"beac9dc55b310ba342766808de57dcc278b26cee9eb495e2a6934382b27c0c24"}
Oct 30 18:46:04.870025 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.869755    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42" event={"ID":"d885b1ae-228c-4843-86e3-99d444486027","Type":"ContainerDied","Data":"a86178aad798a15a5bf429d38426a8829090b01c51ee2035ec53feb3bb5d65dd"}
Oct 30 18:46:04.870025 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.869775    5820 scope.go:122] "RemoveContainer" containerID="beac9dc55b310ba342766808de57dcc278b26cee9eb495e2a6934382b27c0c24"
Oct 30 18:46:04.924504 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.924025    5820 scope.go:122] "RemoveContainer" containerID="beac9dc55b310ba342766808de57dcc278b26cee9eb495e2a6934382b27c0c24"
Oct 30 18:46:04.925546 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:46:04.925514    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"beac9dc55b310ba342766808de57dcc278b26cee9eb495e2a6934382b27c0c24\": container with ID starting with beac9dc55b310ba342766808de57dcc278b26cee9eb495e2a6934382b27c0c24 not found: ID does not exist" containerID="beac9dc55b310ba342766808de57dcc278b26cee9eb495e2a6934382b27c0c24"
Oct 30 18:46:04.925709 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:04.925557    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"beac9dc55b310ba342766808de57dcc278b26cee9eb495e2a6934382b27c0c24"} err="failed to get container status \"beac9dc55b310ba342766808de57dcc278b26cee9eb495e2a6934382b27c0c24\": rpc error: code = NotFound desc = could not find container \"beac9dc55b310ba342766808de57dcc278b26cee9eb495e2a6934382b27c0c24\": container with ID starting with beac9dc55b310ba342766808de57dcc278b26cee9eb495e2a6934382b27c0c24 not found: ID does not exist"
Oct 30 18:46:05.036295 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:05.036254    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"]
Oct 30 18:46:05.044030 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:05.044003    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-529e04e2-7c44d54b47-rcc42"]
Oct 30 18:46:05.435471 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:05.435441    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d885b1ae-228c-4843-86e3-99d444486027" path="/var/lib/kubelet/pods/d885b1ae-228c-4843-86e3-99d444486027/volumes"
Oct 30 18:46:06.075631 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:06.072193    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:06.305608 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:06.305571    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:06.482171 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:06.482097    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:07.457654 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:07.457618    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:07.791183 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:07.790848    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:08.203112 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:46:08.203077    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:46:09.541709 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:09.541678    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"]
Oct 30 18:46:09.542092 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:09.541881    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h" podUID="3a5376da-e0ec-4364-98ed-1079db8ac8f7" containerName="istio-proxy" containerID="cri-o://c2b4485fcc9ec2a3ec572d044f8713db1582517c15b494c5d6e1d286ed509c5d" gracePeriod=30
Oct 30 18:46:09.586422 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:09.586385    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:09.814660 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:09.814616    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:09.950015 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:09.949979    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng"]
Oct 30 18:46:09.950517 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:09.950498    5820 cpu_manager.go:414] "containerMap: removing container" podUID="d885b1ae-228c-4843-86e3-99d444486027" containerName="maas-api"
Oct 30 18:46:09.950628 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:09.950519    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d885b1ae-228c-4843-86e3-99d444486027" containerName="maas-api"
Oct 30 18:46:09.950628 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:09.950553    5820 cpu_manager.go:414] "containerMap: removing container" podUID="89fdf7cd-72bb-4e56-a826-2df992558d41" containerName="payload-processing"
Oct 30 18:46:09.950628 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:09.950571    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89fdf7cd-72bb-4e56-a826-2df992558d41" containerName="payload-processing"
Oct 30 18:46:09.950781 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:09.950693    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="d885b1ae-228c-4843-86e3-99d444486027" containerName="maas-api"
Oct 30 18:46:09.950781 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:09.950710    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="89fdf7cd-72bb-4e56-a826-2df992558d41" containerName="payload-processing"
Oct 30 18:46:09.955347 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:09.955324    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng"
Oct 30 18:46:10.008743 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.008715    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-4a7a52\""
Oct 30 18:46:10.009541 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.008909    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-4a7a52-dockercfg-zbh2k\""
Oct 30 18:46:10.011382 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.011358    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng"]
Oct 30 18:46:10.077363 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.077258    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4km9\" (UniqueName: \"kubernetes.io/projected/91146e5d-893c-4148-aac2-d128b68567de-kube-api-access-n4km9\") pod \"payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng\" (UID: \"91146e5d-893c-4148-aac2-d128b68567de\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng"
Oct 30 18:46:10.077540 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.077394    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91146e5d-893c-4148-aac2-d128b68567de-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng\" (UID: \"91146e5d-893c-4148-aac2-d128b68567de\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng"
Oct 30 18:46:10.178697 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.178661    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4km9\" (UniqueName: \"kubernetes.io/projected/91146e5d-893c-4148-aac2-d128b68567de-kube-api-access-n4km9\") pod \"payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng\" (UID: \"91146e5d-893c-4148-aac2-d128b68567de\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng"
Oct 30 18:46:10.178697 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.178712    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91146e5d-893c-4148-aac2-d128b68567de-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng\" (UID: \"91146e5d-893c-4148-aac2-d128b68567de\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng"
Oct 30 18:46:10.179490 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.179470    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91146e5d-893c-4148-aac2-d128b68567de-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng\" (UID: \"91146e5d-893c-4148-aac2-d128b68567de\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng"
Oct 30 18:46:10.239776 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.239745    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4km9\" (UniqueName: \"kubernetes.io/projected/91146e5d-893c-4148-aac2-d128b68567de-kube-api-access-n4km9\") pod \"payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng\" (UID: \"91146e5d-893c-4148-aac2-d128b68567de\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng"
Oct 30 18:46:10.268134 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.268101    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng"
Oct 30 18:46:10.349943 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.349876    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:10.649435 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.649030    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng"]
Oct 30 18:46:10.652166 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:46:10.652133    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91146e5d_893c_4148_aac2_d128b68567de.slice/crio-629a556e840336f9daf4caf8abd78f977fe41a3852e4c7ddd90715b8499365a0 WatchSource:0}: Error finding container 629a556e840336f9daf4caf8abd78f977fe41a3852e4c7ddd90715b8499365a0: Status 404 returned error can't find the container with id 629a556e840336f9daf4caf8abd78f977fe41a3852e4c7ddd90715b8499365a0
Oct 30 18:46:10.846741 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.846711    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"]
Oct 30 18:46:10.853813 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.853793    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:10.918491 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.918370    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-e7929d-openshift-default-dockercfg-v4s8t\""
Oct 30 18:46:10.919057 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.919015    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"]
Oct 30 18:46:10.927568 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.927395    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng" event={"ID":"91146e5d-893c-4148-aac2-d128b68567de","Type":"ContainerStarted","Data":"15341471d86e65fda0dc3de3042b79b4eac464ee97cb184a9dd3d1903bf45c28"}
Oct 30 18:46:10.927568 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.927456    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng" event={"ID":"91146e5d-893c-4148-aac2-d128b68567de","Type":"ContainerStarted","Data":"629a556e840336f9daf4caf8abd78f977fe41a3852e4c7ddd90715b8499365a0"}
Oct 30 18:46:10.927568 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.927513    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng"
Oct 30 18:46:10.978578 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.978540    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:10.989911 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.989875    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csd5r\" (UniqueName: \"kubernetes.io/projected/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-kube-api-access-csd5r\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:10.990152 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.989929    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istiod-ca-cert\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:10.990152 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.990058    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-workload-certs\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:10.990152 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.990093    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-credential-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:10.990152 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.990119    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-envoy\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:10.990152 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.990147    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-podinfo\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:10.990332 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.990217    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-data\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:10.990332 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.990247    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-workload-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:10.990332 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:10.990302    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-token\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:11.091690 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.091661    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-workload-certs\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:11.091887 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.091699    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-credential-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:11.091887 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.091719    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-envoy\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:11.091887 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.091861    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-podinfo\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:11.092095 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.091998    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-data\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:11.092095 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.092031    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-workload-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:11.092095 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.092075    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-token\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:11.092247 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.092130    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-credential-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:11.092247 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.092176    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-workload-certs\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:11.092354 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.092246    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csd5r\" (UniqueName: \"kubernetes.io/projected/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-kube-api-access-csd5r\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:11.092354 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.092299    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istiod-ca-cert\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:11.092354 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.092340    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-data\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:11.092490 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.092389    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-workload-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:11.092921 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.092904    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istiod-ca-cert\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:11.094830 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.094813    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-envoy\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:11.094983 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.094963    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-podinfo\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:11.106766 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.106716    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng" podStartSLOduration=2.106701204 podStartE2EDuration="2.106701204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:11.105519993 +0000 UTC m=+691.817268690" watchObservedRunningTime="2026-10-30 18:46:11.106701204 +0000 UTC m=+691.818449903"
Oct 30 18:46:11.156162 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.156135    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-token\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:11.156303 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.156186    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-csd5r\" (UniqueName: \"kubernetes.io/projected/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-kube-api-access-csd5r\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:11.166687 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.166666    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:11.197916 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.197878    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:11.387324 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.387291    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:11.570272 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.570192    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:11.597813 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.597784    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"]
Oct 30 18:46:11.600005 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:46:11.599978    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d8121e9_2b0a_4114_ba0e_5b9b8644fb0c.slice/crio-360cc63849a2b6c2ab9e0a69b0a6dcf21874e18366235dbf15965610f3223e32 WatchSource:0}: Error finding container 360cc63849a2b6c2ab9e0a69b0a6dcf21874e18366235dbf15965610f3223e32: Status 404 returned error can't find the container with id 360cc63849a2b6c2ab9e0a69b0a6dcf21874e18366235dbf15965610f3223e32
Oct 30 18:46:11.603270 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.603231    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:46:11.603354 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.603327    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:46:11.603389 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.603376    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:46:11.760539 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.760486    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:11.935725 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.935628    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d" event={"ID":"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c","Type":"ContainerStarted","Data":"5719f491e289737cf0b5b7750f69b79e4bf869a90122f9d13f5a6ecbe0589ad6"}
Oct 30 18:46:11.935725 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:11.935698    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d" event={"ID":"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c","Type":"ContainerStarted","Data":"360cc63849a2b6c2ab9e0a69b0a6dcf21874e18366235dbf15965610f3223e32"}
Oct 30 18:46:12.056903 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:12.056838    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d" podStartSLOduration=2.056817722 podStartE2EDuration="2.056817722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:12.055291549 +0000 UTC m=+692.767040247" watchObservedRunningTime="2026-10-30 18:46:12.056817722 +0000 UTC m=+692.768566424"
Oct 30 18:46:12.174734 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:12.174565    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:12.187280 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:12.187193    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:12.967082 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:12.967021    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:12.968339 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:12.968311    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:14.949035 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:14.940166    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:14.980159 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:14.980123    5820 generic.go:356] "Generic (PLEG): container finished" podID="3a5376da-e0ec-4364-98ed-1079db8ac8f7" containerID="c2b4485fcc9ec2a3ec572d044f8713db1582517c15b494c5d6e1d286ed509c5d" exitCode=0
Oct 30 18:46:14.980357 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:14.980197    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h" event={"ID":"3a5376da-e0ec-4364-98ed-1079db8ac8f7","Type":"ContainerDied","Data":"c2b4485fcc9ec2a3ec572d044f8713db1582517c15b494c5d6e1d286ed509c5d"}
Oct 30 18:46:14.980357 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:14.980282    5820 scope.go:122] "RemoveContainer" containerID="f77cdfc7ac1681d95860448512b5812dcd8f5b642413a8de751f4c0f9a5340d2"
Oct 30 18:46:15.140459 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.131907    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:15.263451 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.263427    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:46:15.360458 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.360411    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:15.363629 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.363601    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-podinfo\") pod \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") "
Oct 30 18:46:15.363779 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.363667    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a5376da-e0ec-4364-98ed-1079db8ac8f7-kube-api-access-gz6mn\" (UniqueName: \"kubernetes.io/projected/3a5376da-e0ec-4364-98ed-1079db8ac8f7-kube-api-access-gz6mn\") pod \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") "
Oct 30 18:46:15.363779 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.363696    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-token\" (UniqueName: \"kubernetes.io/projected/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-token\") pod \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") "
Oct 30 18:46:15.363779 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.363731    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istiod-ca-cert\") pod \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") "
Oct 30 18:46:15.363779 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.363764    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-credential-socket\") pod \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") "
Oct 30 18:46:15.364018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.363808    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-workload-socket\") pod \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") "
Oct 30 18:46:15.364018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.363844    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-envoy\") pod \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") "
Oct 30 18:46:15.364018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.363915    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-data\") pod \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") "
Oct 30 18:46:15.364018 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.364001    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-workload-certs\") pod \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\" (UID: \"3a5376da-e0ec-4364-98ed-1079db8ac8f7\") "
Oct 30 18:46:15.364206 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.364112    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-credential-socket" pod "3a5376da-e0ec-4364-98ed-1079db8ac8f7" (UID: "3a5376da-e0ec-4364-98ed-1079db8ac8f7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:46:15.364451 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.364340    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-workload-socket" pod "3a5376da-e0ec-4364-98ed-1079db8ac8f7" (UID: "3a5376da-e0ec-4364-98ed-1079db8ac8f7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:46:15.364451 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.364369    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-data" pod "3a5376da-e0ec-4364-98ed-1079db8ac8f7" (UID: "3a5376da-e0ec-4364-98ed-1079db8ac8f7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:46:15.364619 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.364497    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:15.364830 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.364797    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-workload-certs" pod "3a5376da-e0ec-4364-98ed-1079db8ac8f7" (UID: "3a5376da-e0ec-4364-98ed-1079db8ac8f7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:46:15.364925 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.364901    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istiod-ca-cert" pod "3a5376da-e0ec-4364-98ed-1079db8ac8f7" (UID: "3a5376da-e0ec-4364-98ed-1079db8ac8f7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:46:15.366313 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.366282    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-podinfo" pod "3a5376da-e0ec-4364-98ed-1079db8ac8f7" (UID: "3a5376da-e0ec-4364-98ed-1079db8ac8f7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:46:15.366313 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.366297    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-token" pod "3a5376da-e0ec-4364-98ed-1079db8ac8f7" (UID: "3a5376da-e0ec-4364-98ed-1079db8ac8f7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:46:15.366487 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.366363    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-envoy" pod "3a5376da-e0ec-4364-98ed-1079db8ac8f7" (UID: "3a5376da-e0ec-4364-98ed-1079db8ac8f7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:46:15.367275 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.367255    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a5376da-e0ec-4364-98ed-1079db8ac8f7-kube-api-access-gz6mn" pod "3a5376da-e0ec-4364-98ed-1079db8ac8f7" (UID: "3a5376da-e0ec-4364-98ed-1079db8ac8f7"). InnerVolumeSpecName "kube-api-access-gz6mn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:46:15.467161 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.467054    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:15.467161 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.467097    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gz6mn\" (UniqueName: \"kubernetes.io/projected/3a5376da-e0ec-4364-98ed-1079db8ac8f7-kube-api-access-gz6mn\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:15.467161 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.467115    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:15.467161 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.467129    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:15.467161 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.467144    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:15.467161 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.467158    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:15.467610 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.467172    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:15.467610 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.467184    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a5376da-e0ec-4364-98ed-1079db8ac8f7-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:15.987475 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.987439    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"
Oct 30 18:46:15.987982 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.987486    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h" event={"ID":"3a5376da-e0ec-4364-98ed-1079db8ac8f7","Type":"ContainerDied","Data":"b201c252fe396d4be55fdccf5155261e08a4e5ddd28c2b62aba0d8a05e2bc664"}
Oct 30 18:46:15.987982 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:15.987553    5820 scope.go:122] "RemoveContainer" containerID="c2b4485fcc9ec2a3ec572d044f8713db1582517c15b494c5d6e1d286ed509c5d"
Oct 30 18:46:16.169916 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:16.169884    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"]
Oct 30 18:46:16.219780 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:16.219725    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-529e04e2-openshift-default-7f66c8f5f7-7nr6h"]
Oct 30 18:46:16.650989 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:16.649746    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:17.460635 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:17.460601    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3a5376da-e0ec-4364-98ed-1079db8ac8f7" path="/var/lib/kubelet/pods/3a5376da-e0ec-4364-98ed-1079db8ac8f7/volumes"
Oct 30 18:46:21.231266 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:21.231184    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:21.795214 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:21.795181    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:21.795403 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:21.795371    5820 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 18:46:21.795454 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:21.795422    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz" podUID="6a253f0b-1803-4c70-b5af-aa295e8c47cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 18:46:21.956974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:21.956120    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng"
Oct 30 18:46:21.984129 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:21.984098    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:22.025944 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.025910    5820 generic.go:356] "Generic (PLEG): container finished" podID="6a253f0b-1803-4c70-b5af-aa295e8c47cb" containerID="1e8c99c5b86a3fde4d5718e0bf838f84c9e76a8dcd5ae60f6cd13e524a2a4e87" exitCode=0
Oct 30 18:46:22.026117 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.026005    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz" event={"ID":"6a253f0b-1803-4c70-b5af-aa295e8c47cb","Type":"ContainerDied","Data":"1e8c99c5b86a3fde4d5718e0bf838f84c9e76a8dcd5ae60f6cd13e524a2a4e87"}
Oct 30 18:46:22.026364 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.026351    5820 scope.go:122] "RemoveContainer" containerID="1e8c99c5b86a3fde4d5718e0bf838f84c9e76a8dcd5ae60f6cd13e524a2a4e87"
Oct 30 18:46:22.027584 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.027558    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:46:22.027639 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.027624    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:46:22.027677 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.027654    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:46:22.308259 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.308230    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz"]
Oct 30 18:46:22.308710 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.308697    5820 cpu_manager.go:414] "containerMap: removing container" podUID="3a5376da-e0ec-4364-98ed-1079db8ac8f7" containerName="istio-proxy"
Oct 30 18:46:22.308754 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.308711    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a5376da-e0ec-4364-98ed-1079db8ac8f7" containerName="istio-proxy"
Oct 30 18:46:22.308754 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.308745    5820 cpu_manager.go:414] "containerMap: removing container" podUID="3a5376da-e0ec-4364-98ed-1079db8ac8f7" containerName="istio-proxy"
Oct 30 18:46:22.308754 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.308751    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a5376da-e0ec-4364-98ed-1079db8ac8f7" containerName="istio-proxy"
Oct 30 18:46:22.308850 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.308812    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a5376da-e0ec-4364-98ed-1079db8ac8f7" containerName="istio-proxy"
Oct 30 18:46:22.308850 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.308822    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a5376da-e0ec-4364-98ed-1079db8ac8f7" containerName="istio-proxy"
Oct 30 18:46:22.311757 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.311729    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz"
Oct 30 18:46:22.371983 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.370479    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz"]
Oct 30 18:46:22.371983 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.370727    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-e7929d\""
Oct 30 18:46:22.401971 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.401242    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-e7929d-dockercfg-djtfh\""
Oct 30 18:46:22.427664 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.425356    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhfct\" (UniqueName: \"kubernetes.io/projected/6579c071-4113-46cb-a9bd-c5d128f57a5b-kube-api-access-hhfct\") pod \"payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz\" (UID: \"6579c071-4113-46cb-a9bd-c5d128f57a5b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz"
Oct 30 18:46:22.427664 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.425467    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6579c071-4113-46cb-a9bd-c5d128f57a5b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz\" (UID: \"6579c071-4113-46cb-a9bd-c5d128f57a5b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz"
Oct 30 18:46:22.526712 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.526665    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhfct\" (UniqueName: \"kubernetes.io/projected/6579c071-4113-46cb-a9bd-c5d128f57a5b-kube-api-access-hhfct\") pod \"payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz\" (UID: \"6579c071-4113-46cb-a9bd-c5d128f57a5b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz"
Oct 30 18:46:22.526905 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.526777    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6579c071-4113-46cb-a9bd-c5d128f57a5b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz\" (UID: \"6579c071-4113-46cb-a9bd-c5d128f57a5b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz"
Oct 30 18:46:22.527509 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.527484    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6579c071-4113-46cb-a9bd-c5d128f57a5b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz\" (UID: \"6579c071-4113-46cb-a9bd-c5d128f57a5b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz"
Oct 30 18:46:22.587888 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.587811    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhfct\" (UniqueName: \"kubernetes.io/projected/6579c071-4113-46cb-a9bd-c5d128f57a5b-kube-api-access-hhfct\") pod \"payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz\" (UID: \"6579c071-4113-46cb-a9bd-c5d128f57a5b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz"
Oct 30 18:46:22.636977 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.635126    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz"
Oct 30 18:46:22.790001 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.769328    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"]
Oct 30 18:46:22.795029 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.794478    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"
Oct 30 18:46:22.801869 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.801840    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:22.809684 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.809654    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"]
Oct 30 18:46:22.829626 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.829508    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d10b9529-eef8-4269-b328-8475335d7ddb-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr\" (UID: \"d10b9529-eef8-4269-b328-8475335d7ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"
Oct 30 18:46:22.829626 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.829568    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d10b9529-eef8-4269-b328-8475335d7ddb-metrics-certs\") pod \"maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr\" (UID: \"d10b9529-eef8-4269-b328-8475335d7ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"
Oct 30 18:46:22.829808 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.829667    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2454\" (UniqueName: \"kubernetes.io/projected/d10b9529-eef8-4269-b328-8475335d7ddb-kube-api-access-n2454\") pod \"maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr\" (UID: \"d10b9529-eef8-4269-b328-8475335d7ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"
Oct 30 18:46:22.829808 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.829729    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d10b9529-eef8-4269-b328-8475335d7ddb-maas-api-tls\") pod \"maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr\" (UID: \"d10b9529-eef8-4269-b328-8475335d7ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"
Oct 30 18:46:22.859026 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.856981    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-4a7a52-serving-cert\""
Oct 30 18:46:22.930803 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.930762    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d10b9529-eef8-4269-b328-8475335d7ddb-metrics-certs\") pod \"maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr\" (UID: \"d10b9529-eef8-4269-b328-8475335d7ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"
Oct 30 18:46:22.930992 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.930859    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2454\" (UniqueName: \"kubernetes.io/projected/d10b9529-eef8-4269-b328-8475335d7ddb-kube-api-access-n2454\") pod \"maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr\" (UID: \"d10b9529-eef8-4269-b328-8475335d7ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"
Oct 30 18:46:22.930992 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.930977    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d10b9529-eef8-4269-b328-8475335d7ddb-maas-api-tls\") pod \"maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr\" (UID: \"d10b9529-eef8-4269-b328-8475335d7ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"
Oct 30 18:46:22.931077 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.931030    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d10b9529-eef8-4269-b328-8475335d7ddb-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr\" (UID: \"d10b9529-eef8-4269-b328-8475335d7ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"
Oct 30 18:46:22.931516 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:46:22.931246    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-serving-cert: secret "maas-api-e2e-worker-w2-4a7a52-serving-cert" not found
Oct 30 18:46:22.931516 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:46:22.931318    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d10b9529-eef8-4269-b328-8475335d7ddb-maas-api-tls podName:d10b9529-eef8-4269-b328-8475335d7ddb nodeName:}" failed. No retries permitted until 2026-10-30 18:46:23.431297471 +0000 UTC m=+704.143046146 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d10b9529-eef8-4269-b328-8475335d7ddb-maas-api-tls") pod "maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr" (UID: "d10b9529-eef8-4269-b328-8475335d7ddb") : secret "maas-api-e2e-worker-w2-4a7a52-serving-cert" not found
Oct 30 18:46:22.931885 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.931840    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d10b9529-eef8-4269-b328-8475335d7ddb-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr\" (UID: \"d10b9529-eef8-4269-b328-8475335d7ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"
Oct 30 18:46:22.971974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.971358    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d10b9529-eef8-4269-b328-8475335d7ddb-metrics-certs\") pod \"maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr\" (UID: \"d10b9529-eef8-4269-b328-8475335d7ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"
Oct 30 18:46:22.989267 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.988869    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:22.992933 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:22.992900    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2454\" (UniqueName: \"kubernetes.io/projected/d10b9529-eef8-4269-b328-8475335d7ddb-kube-api-access-n2454\") pod \"maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr\" (UID: \"d10b9529-eef8-4269-b328-8475335d7ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"
Oct 30 18:46:23.033554 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:23.033525    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz" event={"ID":"6a253f0b-1803-4c70-b5af-aa295e8c47cb","Type":"ContainerStarted","Data":"b777c3733a3fd93aa0d2b8d0118c99b2216857b3f965ee87d745e3f607ebd8ec"}
Oct 30 18:46:23.062415 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:23.062378    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz"]
Oct 30 18:46:23.082532 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:46:23.082485    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6579c071_4113_46cb_a9bd_c5d128f57a5b.slice/crio-2a38a3d40ef9f3d916972ba61a845399c9e92da7cc93988315600bf9300cb4a3 WatchSource:0}: Error finding container 2a38a3d40ef9f3d916972ba61a845399c9e92da7cc93988315600bf9300cb4a3: Status 404 returned error can't find the container with id 2a38a3d40ef9f3d916972ba61a845399c9e92da7cc93988315600bf9300cb4a3
Oct 30 18:46:23.198973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:23.195688    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:23.224536 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:23.224508    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:46:23.224697 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:23.224675    5820 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 18:46:23.224773 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:23.224728    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz" podUID="6a253f0b-1803-4c70-b5af-aa295e8c47cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 18:46:23.361975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:23.361479    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"]
Oct 30 18:46:23.361975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:23.361723    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh" podUID="f11b4ec3-57bb-46a5-8ea4-79183df95ee2" containerName="istio-proxy" containerID="cri-o://6d7aeffdd1b81f6b149b4182807d9dcea48f3511b59e44ab3645a3482e469d79" gracePeriod=30
Oct 30 18:46:23.497976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:23.497523    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d10b9529-eef8-4269-b328-8475335d7ddb-maas-api-tls\") pod \"maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr\" (UID: \"d10b9529-eef8-4269-b328-8475335d7ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"
Oct 30 18:46:23.512556 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:23.512512    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d10b9529-eef8-4269-b328-8475335d7ddb-maas-api-tls\") pod \"maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr\" (UID: \"d10b9529-eef8-4269-b328-8475335d7ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"
Oct 30 18:46:23.514785 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:23.514713    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:23.718871 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:23.718792    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"
Oct 30 18:46:24.090570 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:24.090526    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz" event={"ID":"6579c071-4113-46cb-a9bd-c5d128f57a5b","Type":"ContainerStarted","Data":"1fb5ef5122200d4624ff7683bfd71560789cf521faf0e238b19faf8d479ca18c"}
Oct 30 18:46:24.090745 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:24.090589    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz" event={"ID":"6579c071-4113-46cb-a9bd-c5d128f57a5b","Type":"ContainerStarted","Data":"2a38a3d40ef9f3d916972ba61a845399c9e92da7cc93988315600bf9300cb4a3"}
Oct 30 18:46:24.090993 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:24.090975    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz"
Oct 30 18:46:24.155975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:24.154586    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"]
Oct 30 18:46:24.190245 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:46:24.190158    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd10b9529_eef8_4269_b328_8475335d7ddb.slice/crio-071e57b2d954362e6ea4137fa61e655cfe2a3e0b6cc82bb29378bc7b92ce4f1a WatchSource:0}: Error finding container 071e57b2d954362e6ea4137fa61e655cfe2a3e0b6cc82bb29378bc7b92ce4f1a: Status 404 returned error can't find the container with id 071e57b2d954362e6ea4137fa61e655cfe2a3e0b6cc82bb29378bc7b92ce4f1a
Oct 30 18:46:24.224813 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:24.224786    5820 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 18:46:24.224931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:24.224836    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz" podUID="6a253f0b-1803-4c70-b5af-aa295e8c47cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 18:46:24.259185 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:24.259151    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:24.259460 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:24.259155    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz" podStartSLOduration=2.259135996 podStartE2EDuration="2.259135996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:24.258569104 +0000 UTC m=+704.970317802" watchObservedRunningTime="2026-10-30 18:46:24.259135996 +0000 UTC m=+704.970884695"
Oct 30 18:46:24.915109 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:24.914354    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:25.107099 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:25.107069    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr" event={"ID":"d10b9529-eef8-4269-b328-8475335d7ddb","Type":"ContainerStarted","Data":"071e57b2d954362e6ea4137fa61e655cfe2a3e0b6cc82bb29378bc7b92ce4f1a"}
Oct 30 18:46:25.128677 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:25.128433    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:25.224659 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:25.224620    5820 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 18:46:25.224819 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:25.224671    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz" podUID="6a253f0b-1803-4c70-b5af-aa295e8c47cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 18:46:26.114687 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:26.114651    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr" event={"ID":"d10b9529-eef8-4269-b328-8475335d7ddb","Type":"ContainerStarted","Data":"1a7ba6841253fac6770da89edfdb87bd53861d054bef3adfe13cd0913fd73dfc"}
Oct 30 18:46:26.115131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:26.114836    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"
Oct 30 18:46:26.223998 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:26.223901    5820 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 18:46:26.223998 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:26.223982    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz" podUID="6a253f0b-1803-4c70-b5af-aa295e8c47cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 18:46:26.239976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:26.234491    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr" podStartSLOduration=3.350425831 podStartE2EDuration="4.234474333s" totalImagesPullingTime="884.048502ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:26.232003229 +0000 UTC m=+706.943751927" watchObservedRunningTime="2026-10-30 18:46:26.234474333 +0000 UTC m=+706.946223030"
Oct 30 18:46:27.224191 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:27.224152    5820 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 18:46:27.225288 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:27.225256    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz" podUID="6a253f0b-1803-4c70-b5af-aa295e8c47cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 18:46:28.224587 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.224551    5820 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 18:46:28.225019 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.224607    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz" podUID="6a253f0b-1803-4c70-b5af-aa295e8c47cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 18:46:28.432931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.431742    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:28.659879 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.659842    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:46:28.777311 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.777226    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-credential-socket\") pod \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") "
Oct 30 18:46:28.777311 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.777267    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-envoy\") pod \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") "
Oct 30 18:46:28.777532 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.777317    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-token\" (UniqueName: \"kubernetes.io/projected/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-token\") pod \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") "
Oct 30 18:46:28.777532 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.777344    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-podinfo\") pod \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") "
Oct 30 18:46:28.777532 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.777406    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-workload-socket\") pod \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") "
Oct 30 18:46:28.777532 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.777448    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-kube-api-access-hgt6b\" (UniqueName: \"kubernetes.io/projected/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-kube-api-access-hgt6b\") pod \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") "
Oct 30 18:46:28.777532 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.777477    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-workload-certs\") pod \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") "
Oct 30 18:46:28.777532 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.777513    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-data\") pod \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") "
Oct 30 18:46:28.777837 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.777524    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-credential-socket" pod "f11b4ec3-57bb-46a5-8ea4-79183df95ee2" (UID: "f11b4ec3-57bb-46a5-8ea4-79183df95ee2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:46:28.777837 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.777589    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istiod-ca-cert\") pod \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\" (UID: \"f11b4ec3-57bb-46a5-8ea4-79183df95ee2\") "
Oct 30 18:46:28.778406 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.778020    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-data" pod "f11b4ec3-57bb-46a5-8ea4-79183df95ee2" (UID: "f11b4ec3-57bb-46a5-8ea4-79183df95ee2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:46:28.778406 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.778095    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:28.778406 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.778088    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-workload-certs" pod "f11b4ec3-57bb-46a5-8ea4-79183df95ee2" (UID: "f11b4ec3-57bb-46a5-8ea4-79183df95ee2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:46:28.778406 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.778333    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-workload-socket" pod "f11b4ec3-57bb-46a5-8ea4-79183df95ee2" (UID: "f11b4ec3-57bb-46a5-8ea4-79183df95ee2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:46:28.778716 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.778565    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istiod-ca-cert" pod "f11b4ec3-57bb-46a5-8ea4-79183df95ee2" (UID: "f11b4ec3-57bb-46a5-8ea4-79183df95ee2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:46:28.779906 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.779882    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-podinfo" pod "f11b4ec3-57bb-46a5-8ea4-79183df95ee2" (UID: "f11b4ec3-57bb-46a5-8ea4-79183df95ee2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:46:28.780124 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.780100    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-envoy" pod "f11b4ec3-57bb-46a5-8ea4-79183df95ee2" (UID: "f11b4ec3-57bb-46a5-8ea4-79183df95ee2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:46:28.780273 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.780257    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-kube-api-access-hgt6b" pod "f11b4ec3-57bb-46a5-8ea4-79183df95ee2" (UID: "f11b4ec3-57bb-46a5-8ea4-79183df95ee2"). InnerVolumeSpecName "kube-api-access-hgt6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:46:28.780378 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.780359    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-token" pod "f11b4ec3-57bb-46a5-8ea4-79183df95ee2" (UID: "f11b4ec3-57bb-46a5-8ea4-79183df95ee2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:46:28.879003 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.878936    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:28.879003 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.879006    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:28.879304 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.879021    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:28.879304 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.879035    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:28.879304 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.879050    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:28.879304 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.879063    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgt6b\" (UniqueName: \"kubernetes.io/projected/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-kube-api-access-hgt6b\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:28.879304 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.879078    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:28.879304 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:28.879092    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f11b4ec3-57bb-46a5-8ea4-79183df95ee2-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:29.133119 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:29.133085    5820 generic.go:356] "Generic (PLEG): container finished" podID="f11b4ec3-57bb-46a5-8ea4-79183df95ee2" containerID="6d7aeffdd1b81f6b149b4182807d9dcea48f3511b59e44ab3645a3482e469d79" exitCode=0
Oct 30 18:46:29.133298 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:29.133136    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"
Oct 30 18:46:29.133298 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:29.133158    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh" event={"ID":"f11b4ec3-57bb-46a5-8ea4-79183df95ee2","Type":"ContainerDied","Data":"6d7aeffdd1b81f6b149b4182807d9dcea48f3511b59e44ab3645a3482e469d79"}
Oct 30 18:46:29.133298 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:29.133234    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh" event={"ID":"f11b4ec3-57bb-46a5-8ea4-79183df95ee2","Type":"ContainerDied","Data":"4da7920c82cb7b2503f22e96d3e6d419fdce73d8a460ef7955047448a664251c"}
Oct 30 18:46:29.133298 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:29.133256    5820 scope.go:122] "RemoveContainer" containerID="6d7aeffdd1b81f6b149b4182807d9dcea48f3511b59e44ab3645a3482e469d79"
Oct 30 18:46:29.143938 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:29.143920    5820 scope.go:122] "RemoveContainer" containerID="fc26e0d0be803ba8ecd28ff89e1e8c211650790a9db4a547e34f537a04660658"
Oct 30 18:46:29.153120 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:29.153097    5820 scope.go:122] "RemoveContainer" containerID="6d7aeffdd1b81f6b149b4182807d9dcea48f3511b59e44ab3645a3482e469d79"
Oct 30 18:46:29.153408 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:46:29.153390    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d7aeffdd1b81f6b149b4182807d9dcea48f3511b59e44ab3645a3482e469d79\": container with ID starting with 6d7aeffdd1b81f6b149b4182807d9dcea48f3511b59e44ab3645a3482e469d79 not found: ID does not exist" containerID="6d7aeffdd1b81f6b149b4182807d9dcea48f3511b59e44ab3645a3482e469d79"
Oct 30 18:46:29.153473 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:29.153417    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d7aeffdd1b81f6b149b4182807d9dcea48f3511b59e44ab3645a3482e469d79"} err="failed to get container status \"6d7aeffdd1b81f6b149b4182807d9dcea48f3511b59e44ab3645a3482e469d79\": rpc error: code = NotFound desc = could not find container \"6d7aeffdd1b81f6b149b4182807d9dcea48f3511b59e44ab3645a3482e469d79\": container with ID starting with 6d7aeffdd1b81f6b149b4182807d9dcea48f3511b59e44ab3645a3482e469d79 not found: ID does not exist"
Oct 30 18:46:29.153473 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:29.153436    5820 scope.go:122] "RemoveContainer" containerID="fc26e0d0be803ba8ecd28ff89e1e8c211650790a9db4a547e34f537a04660658"
Oct 30 18:46:29.153677 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:46:29.153664    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc26e0d0be803ba8ecd28ff89e1e8c211650790a9db4a547e34f537a04660658\": container with ID starting with fc26e0d0be803ba8ecd28ff89e1e8c211650790a9db4a547e34f537a04660658 not found: ID does not exist" containerID="fc26e0d0be803ba8ecd28ff89e1e8c211650790a9db4a547e34f537a04660658"
Oct 30 18:46:29.153717 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:29.153681    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc26e0d0be803ba8ecd28ff89e1e8c211650790a9db4a547e34f537a04660658"} err="failed to get container status \"fc26e0d0be803ba8ecd28ff89e1e8c211650790a9db4a547e34f537a04660658\": rpc error: code = NotFound desc = could not find container \"fc26e0d0be803ba8ecd28ff89e1e8c211650790a9db4a547e34f537a04660658\": container with ID starting with fc26e0d0be803ba8ecd28ff89e1e8c211650790a9db4a547e34f537a04660658 not found: ID does not exist"
Oct 30 18:46:29.228507 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:29.228477    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:46:29.229367 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:29.229236    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:46:29.229936 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:29.229913    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 18:46:29.297567 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:29.297534    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"]
Oct 30 18:46:29.347178 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:29.347152    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-620f2843-openshift-default-7bc9dd9457-76szh"]
Oct 30 18:46:29.438260 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:29.438234    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f11b4ec3-57bb-46a5-8ea4-79183df95ee2" path="/var/lib/kubelet/pods/f11b4ec3-57bb-46a5-8ea4-79183df95ee2/volumes"
Oct 30 18:46:29.826315 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:29.826283    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:30.011796 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:30.011761    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:30.140815 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:30.140735    5820 generic.go:356] "Generic (PLEG): container finished" podID="4be0753d-602a-4f4a-a02d-5d6baf686df2" containerID="fb8d8751512c45691d330f0e75dd6a4ab125a2bf91b0f276cfcae2350c47c972" exitCode=0
Oct 30 18:46:30.140973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:30.140814    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6" event={"ID":"4be0753d-602a-4f4a-a02d-5d6baf686df2","Type":"ContainerDied","Data":"fb8d8751512c45691d330f0e75dd6a4ab125a2bf91b0f276cfcae2350c47c972"}
Oct 30 18:46:30.141257 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:30.141239    5820 scope.go:122] "RemoveContainer" containerID="fb8d8751512c45691d330f0e75dd6a4ab125a2bf91b0f276cfcae2350c47c972"
Oct 30 18:46:30.142502 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:30.142476    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:46:30.142560 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:30.142545    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:46:30.142594 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:30.142574    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:46:30.206460 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:30.206419    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:31.165258 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:31.165220    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6" event={"ID":"4be0753d-602a-4f4a-a02d-5d6baf686df2","Type":"ContainerStarted","Data":"157cc81bc9ef32c0907cd568640071754e0533cf660713eeeca407afa24f63da"}
Oct 30 18:46:31.795142 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:31.795109    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:46:31.795538 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:31.795516    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Oct 30 18:46:31.795631 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:31.795562    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6" podUID="4be0753d-602a-4f4a-a02d-5d6baf686df2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Oct 30 18:46:32.125849 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:32.125815    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"
Oct 30 18:46:32.796290 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:32.796257    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Oct 30 18:46:32.796706 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:32.796315    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6" podUID="4be0753d-602a-4f4a-a02d-5d6baf686df2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Oct 30 18:46:33.121773 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.121734    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:33.358977 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.357910    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:33.654971 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.647208    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx"]
Oct 30 18:46:33.654971 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.648041    5820 cpu_manager.go:414] "containerMap: removing container" podUID="f11b4ec3-57bb-46a5-8ea4-79183df95ee2" containerName="istio-proxy"
Oct 30 18:46:33.654971 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.648060    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f11b4ec3-57bb-46a5-8ea4-79183df95ee2" containerName="istio-proxy"
Oct 30 18:46:33.654971 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.648077    5820 cpu_manager.go:414] "containerMap: removing container" podUID="f11b4ec3-57bb-46a5-8ea4-79183df95ee2" containerName="istio-proxy"
Oct 30 18:46:33.654971 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.648085    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f11b4ec3-57bb-46a5-8ea4-79183df95ee2" containerName="istio-proxy"
Oct 30 18:46:33.654971 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.648185    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="f11b4ec3-57bb-46a5-8ea4-79183df95ee2" containerName="istio-proxy"
Oct 30 18:46:33.669068 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.665394    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx"
Oct 30 18:46:33.709184 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.709153    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx"]
Oct 30 18:46:33.729122 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.729089    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwv6h\" (UniqueName: \"kubernetes.io/projected/b576ab89-f39f-49d9-964b-333d324a9c3d-kube-api-access-vwv6h\") pod \"payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx\" (UID: \"b576ab89-f39f-49d9-964b-333d324a9c3d\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx"
Oct 30 18:46:33.729280 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.729141    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b576ab89-f39f-49d9-964b-333d324a9c3d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx\" (UID: \"b576ab89-f39f-49d9-964b-333d324a9c3d\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx"
Oct 30 18:46:33.763976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.761357    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-fd61e3-dockercfg-56g4x\""
Oct 30 18:46:33.763976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.761455    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-fd61e3\""
Oct 30 18:46:33.795624 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.795591    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Oct 30 18:46:33.795802 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.795640    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6" podUID="4be0753d-602a-4f4a-a02d-5d6baf686df2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Oct 30 18:46:33.798318 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.798267    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:33.830690 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.830652    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwv6h\" (UniqueName: \"kubernetes.io/projected/b576ab89-f39f-49d9-964b-333d324a9c3d-kube-api-access-vwv6h\") pod \"payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx\" (UID: \"b576ab89-f39f-49d9-964b-333d324a9c3d\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx"
Oct 30 18:46:33.830886 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.830695    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b576ab89-f39f-49d9-964b-333d324a9c3d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx\" (UID: \"b576ab89-f39f-49d9-964b-333d324a9c3d\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx"
Oct 30 18:46:33.831451 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.831434    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b576ab89-f39f-49d9-964b-333d324a9c3d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx\" (UID: \"b576ab89-f39f-49d9-964b-333d324a9c3d\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx"
Oct 30 18:46:33.850976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.846293    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw"]
Oct 30 18:46:33.859180 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.856245    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="f11b4ec3-57bb-46a5-8ea4-79183df95ee2" containerName="istio-proxy"
Oct 30 18:46:33.862262 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.862237    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw"
Oct 30 18:46:33.892481 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.892449    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwv6h\" (UniqueName: \"kubernetes.io/projected/b576ab89-f39f-49d9-964b-333d324a9c3d-kube-api-access-vwv6h\") pod \"payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx\" (UID: \"b576ab89-f39f-49d9-964b-333d324a9c3d\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx"
Oct 30 18:46:33.912732 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.912662    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw"]
Oct 30 18:46:33.931717 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.931689    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnwtv\" (UniqueName: \"kubernetes.io/projected/c01e0998-f450-41a4-be3f-c9a94ea6ff59-kube-api-access-wnwtv\") pod \"payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw\" (UID: \"c01e0998-f450-41a4-be3f-c9a94ea6ff59\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw"
Oct 30 18:46:33.931927 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.931903    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c01e0998-f450-41a4-be3f-c9a94ea6ff59-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw\" (UID: \"c01e0998-f450-41a4-be3f-c9a94ea6ff59\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw"
Oct 30 18:46:33.992683 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:33.992653    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx"
Oct 30 18:46:34.032607 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:34.032568    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c01e0998-f450-41a4-be3f-c9a94ea6ff59-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw\" (UID: \"c01e0998-f450-41a4-be3f-c9a94ea6ff59\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw"
Oct 30 18:46:34.032788 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:34.032630    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnwtv\" (UniqueName: \"kubernetes.io/projected/c01e0998-f450-41a4-be3f-c9a94ea6ff59-kube-api-access-wnwtv\") pod \"payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw\" (UID: \"c01e0998-f450-41a4-be3f-c9a94ea6ff59\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw"
Oct 30 18:46:34.033319 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:34.033301    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c01e0998-f450-41a4-be3f-c9a94ea6ff59-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw\" (UID: \"c01e0998-f450-41a4-be3f-c9a94ea6ff59\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw"
Oct 30 18:46:34.094912 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:34.094877    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnwtv\" (UniqueName: \"kubernetes.io/projected/c01e0998-f450-41a4-be3f-c9a94ea6ff59-kube-api-access-wnwtv\") pod \"payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw\" (UID: \"c01e0998-f450-41a4-be3f-c9a94ea6ff59\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw"
Oct 30 18:46:34.177179 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:34.177091    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw"
Oct 30 18:46:34.391606 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:34.391585    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx"]
Oct 30 18:46:34.393211 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:46:34.393181    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb576ab89_f39f_49d9_964b_333d324a9c3d.slice/crio-a7255ddf87efc06dafb6eb57c53814752d98bed66a29a4073862b6a7375dccfa WatchSource:0}: Error finding container a7255ddf87efc06dafb6eb57c53814752d98bed66a29a4073862b6a7375dccfa: Status 404 returned error can't find the container with id a7255ddf87efc06dafb6eb57c53814752d98bed66a29a4073862b6a7375dccfa
Oct 30 18:46:34.560071 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:34.559259    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw"]
Oct 30 18:46:34.563410 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:46:34.563385    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc01e0998_f450_41a4_be3f_c9a94ea6ff59.slice/crio-845396b929b964c935a2b52f8f7b8e8bd2eca0b5a6a828e03f77ccfb9e421ee0 WatchSource:0}: Error finding container 845396b929b964c935a2b52f8f7b8e8bd2eca0b5a6a828e03f77ccfb9e421ee0: Status 404 returned error can't find the container with id 845396b929b964c935a2b52f8f7b8e8bd2eca0b5a6a828e03f77ccfb9e421ee0
Oct 30 18:46:34.795716 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:34.795634    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Oct 30 18:46:34.795716 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:34.795681    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6" podUID="4be0753d-602a-4f4a-a02d-5d6baf686df2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Oct 30 18:46:35.190223 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:35.190188    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw" event={"ID":"c01e0998-f450-41a4-be3f-c9a94ea6ff59","Type":"ContainerStarted","Data":"9f1993aab2fff61086fab35dd9129d09178456f55764b94d5753e692642bd072"}
Oct 30 18:46:35.190699 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:35.190253    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw" event={"ID":"c01e0998-f450-41a4-be3f-c9a94ea6ff59","Type":"ContainerStarted","Data":"845396b929b964c935a2b52f8f7b8e8bd2eca0b5a6a828e03f77ccfb9e421ee0"}
Oct 30 18:46:35.190699 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:35.190373    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw"
Oct 30 18:46:35.192015 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:35.191995    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx" event={"ID":"b576ab89-f39f-49d9-964b-333d324a9c3d","Type":"ContainerStarted","Data":"7f959ec52dc1a86d1c9688db7f0b34c5af5015db283923081a456259c3445368"}
Oct 30 18:46:35.192102 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:35.192027    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx" event={"ID":"b576ab89-f39f-49d9-964b-333d324a9c3d","Type":"ContainerStarted","Data":"a7255ddf87efc06dafb6eb57c53814752d98bed66a29a4073862b6a7375dccfa"}
Oct 30 18:46:35.192182 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:35.192168    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx"
Oct 30 18:46:35.322507 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:35.322453    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw" podStartSLOduration=2.322440845 podStartE2EDuration="2.322440845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:35.320694555 +0000 UTC m=+716.032443264" watchObservedRunningTime="2026-10-30 18:46:35.322440845 +0000 UTC m=+716.034189536"
Oct 30 18:46:35.796092 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:35.796058    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Oct 30 18:46:35.796291 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:35.796111    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6" podUID="4be0753d-602a-4f4a-a02d-5d6baf686df2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Oct 30 18:46:36.118068 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:36.118040    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz"
Oct 30 18:46:36.236622 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:36.236572    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx" podStartSLOduration=3.236557015 podStartE2EDuration="3.236557015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:35.445383476 +0000 UTC m=+716.157132174" watchObservedRunningTime="2026-10-30 18:46:36.236557015 +0000 UTC m=+716.948305711"
Oct 30 18:46:36.799481 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:36.799444    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:46:36.799745 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:36.799720    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:46:36.800442 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:36.800426    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 18:46:38.203248 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:46:38.203201    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:46:38.549405 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:38.549378    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"]
Oct 30 18:46:38.553246 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:38.553228    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"
Oct 30 18:46:38.605376 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:38.605351    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-e7929d-serving-cert\""
Oct 30 18:46:38.611453 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:38.611430    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"]
Oct 30 18:46:38.613639 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:38.613614    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27cbbf2c-1014-49b5-86b8-de8041a0b268-metrics-certs\") pod \"maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4\" (UID: \"27cbbf2c-1014-49b5-86b8-de8041a0b268\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"
Oct 30 18:46:38.613728 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:38.613665    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27cbbf2c-1014-49b5-86b8-de8041a0b268-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4\" (UID: \"27cbbf2c-1014-49b5-86b8-de8041a0b268\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"
Oct 30 18:46:38.613820 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:38.613797    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf8hz\" (UniqueName: \"kubernetes.io/projected/27cbbf2c-1014-49b5-86b8-de8041a0b268-kube-api-access-bf8hz\") pod \"maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4\" (UID: \"27cbbf2c-1014-49b5-86b8-de8041a0b268\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"
Oct 30 18:46:38.613860 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:38.613839    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27cbbf2c-1014-49b5-86b8-de8041a0b268-maas-api-tls\") pod \"maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4\" (UID: \"27cbbf2c-1014-49b5-86b8-de8041a0b268\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"
Oct 30 18:46:38.714737 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:38.714709    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf8hz\" (UniqueName: \"kubernetes.io/projected/27cbbf2c-1014-49b5-86b8-de8041a0b268-kube-api-access-bf8hz\") pod \"maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4\" (UID: \"27cbbf2c-1014-49b5-86b8-de8041a0b268\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"
Oct 30 18:46:38.714737 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:38.714742    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27cbbf2c-1014-49b5-86b8-de8041a0b268-maas-api-tls\") pod \"maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4\" (UID: \"27cbbf2c-1014-49b5-86b8-de8041a0b268\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"
Oct 30 18:46:38.714941 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:38.714813    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27cbbf2c-1014-49b5-86b8-de8041a0b268-metrics-certs\") pod \"maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4\" (UID: \"27cbbf2c-1014-49b5-86b8-de8041a0b268\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"
Oct 30 18:46:38.714941 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:38.714838    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27cbbf2c-1014-49b5-86b8-de8041a0b268-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4\" (UID: \"27cbbf2c-1014-49b5-86b8-de8041a0b268\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"
Oct 30 18:46:38.714941 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:46:38.714917    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-serving-cert: secret "maas-api-e2e-worker-w4-e7929d-serving-cert" not found
Oct 30 18:46:38.715078 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:46:38.714990    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27cbbf2c-1014-49b5-86b8-de8041a0b268-maas-api-tls podName:27cbbf2c-1014-49b5-86b8-de8041a0b268 nodeName:}" failed. No retries permitted until 2026-10-30 18:46:39.214975898 +0000 UTC m=+719.926724586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/27cbbf2c-1014-49b5-86b8-de8041a0b268-maas-api-tls") pod "maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4" (UID: "27cbbf2c-1014-49b5-86b8-de8041a0b268") : secret "maas-api-e2e-worker-w4-e7929d-serving-cert" not found
Oct 30 18:46:38.715545 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:38.715521    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27cbbf2c-1014-49b5-86b8-de8041a0b268-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4\" (UID: \"27cbbf2c-1014-49b5-86b8-de8041a0b268\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"
Oct 30 18:46:38.717358 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:38.717342    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27cbbf2c-1014-49b5-86b8-de8041a0b268-metrics-certs\") pod \"maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4\" (UID: \"27cbbf2c-1014-49b5-86b8-de8041a0b268\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"
Oct 30 18:46:38.775656 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:38.775631    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf8hz\" (UniqueName: \"kubernetes.io/projected/27cbbf2c-1014-49b5-86b8-de8041a0b268-kube-api-access-bf8hz\") pod \"maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4\" (UID: \"27cbbf2c-1014-49b5-86b8-de8041a0b268\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"
Oct 30 18:46:38.784108 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:38.784080    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:38.982994 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:38.982874    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:39.162120 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:39.162088    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:39.219930 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:39.219901    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27cbbf2c-1014-49b5-86b8-de8041a0b268-maas-api-tls\") pod \"maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4\" (UID: \"27cbbf2c-1014-49b5-86b8-de8041a0b268\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"
Oct 30 18:46:39.223190 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:39.223162    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27cbbf2c-1014-49b5-86b8-de8041a0b268-maas-api-tls\") pod \"maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4\" (UID: \"27cbbf2c-1014-49b5-86b8-de8041a0b268\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"
Oct 30 18:46:39.346315 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:39.346271    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:39.467040 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:39.467013    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"
Oct 30 18:46:39.850934 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:39.850903    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"]
Oct 30 18:46:39.852801 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:46:39.852772    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27cbbf2c_1014_49b5_86b8_de8041a0b268.slice/crio-1a036e5d35121683e26e65289535c3a783610ad187fa5e43d802fc4f305f5420 WatchSource:0}: Error finding container 1a036e5d35121683e26e65289535c3a783610ad187fa5e43d802fc4f305f5420: Status 404 returned error can't find the container with id 1a036e5d35121683e26e65289535c3a783610ad187fa5e43d802fc4f305f5420
Oct 30 18:46:39.876064 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:39.876039    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:40.103136 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:40.103047    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:40.229622 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:40.229587    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4" event={"ID":"27cbbf2c-1014-49b5-86b8-de8041a0b268","Type":"ContainerStarted","Data":"1a036e5d35121683e26e65289535c3a783610ad187fa5e43d802fc4f305f5420"}
Oct 30 18:46:40.282263 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:40.282226    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:40.919899 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:40.919865    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:41.116057 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:41.116016    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:41.236846 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:41.236804    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4" event={"ID":"27cbbf2c-1014-49b5-86b8-de8041a0b268","Type":"ContainerStarted","Data":"eeb054b9352526886aca6d37dea8aab52f4a8ea0a3ac5ca21618ce8495dccf88"}
Oct 30 18:46:41.237300 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:41.236875    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"
Oct 30 18:46:41.354942 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:41.354892    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4" podStartSLOduration=2.521619962 podStartE2EDuration="3.354878072s" totalImagesPullingTime="833.25811ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:41.354131386 +0000 UTC m=+722.065880083" watchObservedRunningTime="2026-10-30 18:46:41.354878072 +0000 UTC m=+722.066626768"
Oct 30 18:46:44.146051 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:44.146008    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:44.332160 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:44.332129    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:46.202998 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:46.202932    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw"
Oct 30 18:46:46.210776 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:46.210744    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx"
Oct 30 18:46:47.248072 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:47.247988    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"
Oct 30 18:46:49.153632 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:49.153539    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:50.263661 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:50.263621    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:50.455796 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:50.455750    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:53.868879 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:53.868841    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:54.044941 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:54.044902    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:54.223170 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:54.223086    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:57.327043 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:57.327006    5820 generic.go:356] "Generic (PLEG): container finished" podID="7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" containerID="5719f491e289737cf0b5b7750f69b79e4bf869a90122f9d13f5a6ecbe0589ad6" exitCode=0
Oct 30 18:46:57.327442 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:57.327085    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d" event={"ID":"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c","Type":"ContainerDied","Data":"5719f491e289737cf0b5b7750f69b79e4bf869a90122f9d13f5a6ecbe0589ad6"}
Oct 30 18:46:57.327488 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:57.327477    5820 scope.go:122] "RemoveContainer" containerID="5719f491e289737cf0b5b7750f69b79e4bf869a90122f9d13f5a6ecbe0589ad6"
Oct 30 18:46:57.328830 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:57.328800    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:46:57.328889 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:57.328873    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:46:57.328928 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:57.328911    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:46:57.966958 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:57.966898    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:58.334198 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:58.334167    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d" event={"ID":"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c","Type":"ContainerStarted","Data":"e482831028717aa37ecb4c2c577a2b3fcd6bde8c3aaf1348bdfd5ffc6ac7d3a5"}
Oct 30 18:46:58.963459 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:58.963414    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:59.167301 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:59.167258    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:46:59.168611 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:59.168589    5820 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Oct 30 18:46:59.168678 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:59.168651    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d" podUID="7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Oct 30 18:46:59.171076 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:59.171045    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:59.603595 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:59.603560    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:59.860374 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:46:59.860263    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:00.168134 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:00.168059    5820 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Oct 30 18:47:00.168134 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:00.168114    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d" podUID="7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Oct 30 18:47:01.167270 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:01.167237    5820 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Oct 30 18:47:01.167702 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:01.167299    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d" podUID="7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Oct 30 18:47:02.167538 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:02.167457    5820 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Oct 30 18:47:02.167538 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:02.167510    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d" podUID="7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Oct 30 18:47:03.167710 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:03.167673    5820 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Oct 30 18:47:03.168155 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:03.167725    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d" podUID="7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Oct 30 18:47:04.172022 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:04.171992    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:47:04.172464 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:04.172244    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:47:04.173000 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:04.172978    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 18:47:04.258377 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:04.258344    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:06.208157 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:06.208127    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:08.202848 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:47:08.202809    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:47:09.045836 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:09.045798    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:13.377973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:13.377809    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:13.968255 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:13.968218    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:17.190080 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:17.190042    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:17.889880 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:17.889848    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:18.077902 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:18.077866    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:19.126151 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:19.126116    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:23.931299 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:23.931202    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:28.730770 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:28.730730    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:30.382162 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:30.382119    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:31.064529 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:31.064492    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:31.251944 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:31.251905    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:33.838711 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:33.838529    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:38.202819 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:47:38.202786    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:47:38.757076 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:38.757038    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:39.509227 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:39.509192    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:43.599102 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:43.599069    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:44.788323 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.788290    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"]
Oct 30 18:47:44.792916 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.792892    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:44.853780 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.853745    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-tmp-dir\") pod \"simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") " pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:44.853964 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.853807    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-tls-certs\") pod \"simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") " pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:44.853964 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.853900    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-dshm\") pod \"simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") " pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:44.854086 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.853965    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqf5h\" (UniqueName: \"kubernetes.io/projected/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-kube-api-access-cqf5h\") pod \"simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") " pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:44.854151 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.854104    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-home\") pod \"simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") " pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:44.854151 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.854146    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-model-cache\") pod \"simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") " pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:44.896631 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.896603    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-e7929d\"/\"openshift-service-ca.crt\""
Oct 30 18:47:44.896826 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.896670    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-e7929d\"/\"kube-root-ca.crt\""
Oct 30 18:47:44.896826 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.896767    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-e7929d\"/\"simulator-w4-e7929d-kserve-self-signed-certs\""
Oct 30 18:47:44.896826 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.896812    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-e7929d\"/\"default-dockercfg-h7dfc\""
Oct 30 18:47:44.898648 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.898623    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"]
Oct 30 18:47:44.927364 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.927259    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:44.955657 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.955624    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-model-cache\") pod \"simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") " pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:44.955842 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.955664    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-tmp-dir\") pod \"simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") " pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:44.955842 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.955698    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-tls-certs\") pod \"simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") " pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:44.955842 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.955729    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-dshm\") pod \"simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") " pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:44.955842 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.955765    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqf5h\" (UniqueName: \"kubernetes.io/projected/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-kube-api-access-cqf5h\") pod \"simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") " pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:44.956034 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.955851    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-home\") pod \"simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") " pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:44.956149 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.956127    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-tmp-dir\") pod \"simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") " pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:44.956193 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.956150    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-model-cache\") pod \"simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") " pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:44.956227 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.956209    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-home\") pod \"simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") " pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:44.958209 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.958183    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-dshm\") pod \"simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") " pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:44.958453 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:44.958436    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-tls-certs\") pod \"simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") " pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:45.016594 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:45.016564    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqf5h\" (UniqueName: \"kubernetes.io/projected/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-kube-api-access-cqf5h\") pod \"simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") " pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:45.116390 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:45.116352    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:45.494084 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:45.494057    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"]
Oct 30 18:47:45.495153 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:47:45.495128    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d33ec3b_bb2f_428f_86dd_42ad3c7009d0.slice/crio-bf3b18f4d10a7c30ba0066a793908586f54f065993818db58170ca3a8e55a35c WatchSource:0}: Error finding container bf3b18f4d10a7c30ba0066a793908586f54f065993818db58170ca3a8e55a35c: Status 404 returned error can't find the container with id bf3b18f4d10a7c30ba0066a793908586f54f065993818db58170ca3a8e55a35c
Oct 30 18:47:45.613226 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:45.613187    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw" event={"ID":"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0","Type":"ContainerStarted","Data":"3fdd16eb7b7436ad0fb2da1ef90eae602c3cd4198c0cdeb347ccfaad0b519541"}
Oct 30 18:47:45.613396 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:45.613242    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw" event={"ID":"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0","Type":"ContainerStarted","Data":"bf3b18f4d10a7c30ba0066a793908586f54f065993818db58170ca3a8e55a35c"}
Oct 30 18:47:45.730483 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:45.730423    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw" podStartSLOduration=1.730407442 podStartE2EDuration="1.730407442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:47:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:45.72889579 +0000 UTC m=+786.440644490" watchObservedRunningTime="2026-10-30 18:47:45.730407442 +0000 UTC m=+786.442156168"
Oct 30 18:47:46.116984 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:46.116943    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:48.400378 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:48.396469    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:50.604905 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:50.604871    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:53.162193 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:53.162136    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:55.116993 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:55.116935    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:55.129387 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:55.129356    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:55.675055 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:55.675028    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:47:56.298657 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:56.298566    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:57.930855 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:47:57.930690    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:02.727277 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:02.727220    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:07.625683 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:07.625650    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:08.205349 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:48:08.205306    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:48:11.135618 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:11.135571    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:12.284245 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:12.283021    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:17.133684 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:17.133598    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:21.901850 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:21.901795    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:24.722890 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:24.722843    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:26.665900 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:26.665861    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:31.454886 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:31.454851    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:36.258520 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:36.258479    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:38.202804 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:48:38.202768    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:48:41.061850 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:41.061816    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:44.711004 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.710969    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"]
Oct 30 18:48:44.715348 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.715326    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:44.789130 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.789096    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-home\") pod \"premium-w4-e7929d-kserve-8644b74cbd-nwv2b\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") " pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:44.789325 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.789141    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-model-cache\") pod \"premium-w4-e7929d-kserve-8644b74cbd-nwv2b\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") " pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:44.789325 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.789166    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-dshm\") pod \"premium-w4-e7929d-kserve-8644b74cbd-nwv2b\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") " pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:44.789325 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.789292    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-tmp-dir\") pod \"premium-w4-e7929d-kserve-8644b74cbd-nwv2b\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") " pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:44.789518 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.789328    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj84j\" (UniqueName: \"kubernetes.io/projected/1cae2237-0916-415a-9059-b94e0d950b0a-kube-api-access-mj84j\") pod \"premium-w4-e7929d-kserve-8644b74cbd-nwv2b\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") " pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:44.789518 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.789464    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cae2237-0916-415a-9059-b94e0d950b0a-tls-certs\") pod \"premium-w4-e7929d-kserve-8644b74cbd-nwv2b\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") " pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:44.820218 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.820188    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-e7929d\"/\"premium-w4-e7929d-kserve-self-signed-certs\""
Oct 30 18:48:44.822276 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.822255    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"]
Oct 30 18:48:44.897705 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.897670    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-dshm\") pod \"premium-w4-e7929d-kserve-8644b74cbd-nwv2b\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") " pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:44.897895 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.897813    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-tmp-dir\") pod \"premium-w4-e7929d-kserve-8644b74cbd-nwv2b\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") " pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:44.897895 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.897852    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj84j\" (UniqueName: \"kubernetes.io/projected/1cae2237-0916-415a-9059-b94e0d950b0a-kube-api-access-mj84j\") pod \"premium-w4-e7929d-kserve-8644b74cbd-nwv2b\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") " pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:44.898063 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.897921    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cae2237-0916-415a-9059-b94e0d950b0a-tls-certs\") pod \"premium-w4-e7929d-kserve-8644b74cbd-nwv2b\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") " pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:44.898063 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.897997    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-home\") pod \"premium-w4-e7929d-kserve-8644b74cbd-nwv2b\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") " pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:44.898063 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.898044    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-model-cache\") pod \"premium-w4-e7929d-kserve-8644b74cbd-nwv2b\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") " pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:44.899422 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.898655    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-model-cache\") pod \"premium-w4-e7929d-kserve-8644b74cbd-nwv2b\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") " pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:44.900112 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.899916    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-tmp-dir\") pod \"premium-w4-e7929d-kserve-8644b74cbd-nwv2b\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") " pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:44.900112 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.900058    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-home\") pod \"premium-w4-e7929d-kserve-8644b74cbd-nwv2b\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") " pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:44.916613 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.915293    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-dshm\") pod \"premium-w4-e7929d-kserve-8644b74cbd-nwv2b\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") " pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:44.917210 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.917178    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:44.922734 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.922672    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cae2237-0916-415a-9059-b94e0d950b0a-tls-certs\") pod \"premium-w4-e7929d-kserve-8644b74cbd-nwv2b\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") " pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:44.960964 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:44.960914    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj84j\" (UniqueName: \"kubernetes.io/projected/1cae2237-0916-415a-9059-b94e0d950b0a-kube-api-access-mj84j\") pod \"premium-w4-e7929d-kserve-8644b74cbd-nwv2b\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") " pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:45.027142 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:45.027051    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:45.426527 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:45.426502    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"]
Oct 30 18:48:45.428535 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:48:45.428507    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1cae2237_0916_415a_9059_b94e0d950b0a.slice/crio-639bcd2e050e734adc79c03e0a5ae69fb4e408eb8b914e9fa68a60d5d68b20b2 WatchSource:0}: Error finding container 639bcd2e050e734adc79c03e0a5ae69fb4e408eb8b914e9fa68a60d5d68b20b2: Status 404 returned error can't find the container with id 639bcd2e050e734adc79c03e0a5ae69fb4e408eb8b914e9fa68a60d5d68b20b2
Oct 30 18:48:45.939465 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:45.939429    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b" event={"ID":"1cae2237-0916-415a-9059-b94e0d950b0a","Type":"ContainerStarted","Data":"423139a8afde64a3b1ab746212e7a52fa160319f26adb1d696cb35b568e8486f"}
Oct 30 18:48:45.939862 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:45.939481    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b" event={"ID":"1cae2237-0916-415a-9059-b94e0d950b0a","Type":"ContainerStarted","Data":"639bcd2e050e734adc79c03e0a5ae69fb4e408eb8b914e9fa68a60d5d68b20b2"}
Oct 30 18:48:45.966042 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:45.965988    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:46.027773 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:46.027741    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:46.062138 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:46.062088    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b" podStartSLOduration=2.06207268 podStartE2EDuration="2.06207268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:46.058291638 +0000 UTC m=+846.770040334" watchObservedRunningTime="2026-10-30 18:48:46.06207268 +0000 UTC m=+846.773821380"
Oct 30 18:48:50.980769 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:50.980720    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:51.288605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:51.288515    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:53.879209 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:53.879173    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:55.034885 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:55.034856    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:55.075639 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:55.075608    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:48:55.748593 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:55.748561    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:56.003211 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:48:56.003139    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:49:00.549828 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:49:00.549787    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:05.366607 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:49:05.366570    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:08.202854 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:49:08.202817    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:49:10.275362 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:49:10.275325    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:10.668222 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:49:10.667478    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:15.071814 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:49:15.071776    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:19.891054 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:49:19.891015    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:24.847894 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:49:24.847860    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:29.540049 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:49:29.539986    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:34.433721 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:49:34.433487    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:38.202453 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:49:38.202412    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:49:39.350691 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:49:39.350642    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:44.244581 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:49:44.244527    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:49.053173 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:49:49.053078    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:51.108109 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:49:51.108071    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:51.306485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:49:51.306443    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:51.551772 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:49:51.551728    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:53.984483 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:49:53.984446    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:55.365233 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:49:55.365196    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:58.769064 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:49:58.768976    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:00.537397 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:00.537343    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:00.744301 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:00.744246    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:03.697491 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:03.697457    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:08.203597 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:50:08.203562    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:50:08.643240 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:08.643177    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:09.215905 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.215873    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"]
Oct 30 18:50:09.219855 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.219832    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.272119 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.272094    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"]
Oct 30 18:50:09.275891 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.275865    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.321659 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.321635    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"]
Oct 30 18:50:09.343180 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.343142    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.343316 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.343209    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.343316 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.343244    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.343316 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.343278    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.343433 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.343322    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.343433 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.343413    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trnkw\" (UniqueName: \"kubernetes.io/projected/76921ad3-c8c7-4de8-b9ef-5198a8c94327-kube-api-access-trnkw\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.343573 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.343556    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.343635 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.343620    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.343688 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.343674    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.392548 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.392516    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"]
Oct 30 18:50:09.445061 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.445025    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.445061 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.445061    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.445300 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.445085    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.445300 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.445154    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.445300 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.445275    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.445472 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.445319    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trnkw\" (UniqueName: \"kubernetes.io/projected/76921ad3-c8c7-4de8-b9ef-5198a8c94327-kube-api-access-trnkw\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.445472 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.445342    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.445472 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.445398    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.445472 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.445456    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.445683 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.445485    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.445683 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.445508    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.445683 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.445510    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.445683 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.445540    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.445683 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.445636    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.445683 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.445669    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a373b999-a0cb-4016-9415-e977bd058b1c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.445905 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.445755    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a373b999-a0cb-4016-9415-e977bd058b1c-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.445905 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.445770    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.445905 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.445859    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.445905 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.445897    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.446159 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.445906    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.446159 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.445933    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz2qw\" (UniqueName: \"kubernetes.io/projected/a373b999-a0cb-4016-9415-e977bd058b1c-kube-api-access-fz2qw\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.446159 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.445985    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a373b999-a0cb-4016-9415-e977bd058b1c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.446279 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.446177    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.447965 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.447928    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.448285 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.448266    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.506639 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.506576    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.506758 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.506643    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trnkw\" (UniqueName: \"kubernetes.io/projected/76921ad3-c8c7-4de8-b9ef-5198a8c94327-kube-api-access-trnkw\") pod \"maas-default-gateway-openshift-default-66db998887-qxlbf\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.533126 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.533100    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:09.546697 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.546663    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz2qw\" (UniqueName: \"kubernetes.io/projected/a373b999-a0cb-4016-9415-e977bd058b1c-kube-api-access-fz2qw\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.546780 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.546709    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a373b999-a0cb-4016-9415-e977bd058b1c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.546780 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.546756    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.546878 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.546839    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.546935 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.546886    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.547017 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.546938    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.547017 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.546995    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.547120 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.547043    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a373b999-a0cb-4016-9415-e977bd058b1c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.547180 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.547123    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a373b999-a0cb-4016-9415-e977bd058b1c-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.547285 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.547262    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.547560 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.547533    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.548131 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.548091    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.548234 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.548135    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.548406 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.548380    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a373b999-a0cb-4016-9415-e977bd058b1c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.550064 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.550042    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.550166 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.550091    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a373b999-a0cb-4016-9415-e977bd058b1c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.610284 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.610260    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a373b999-a0cb-4016-9415-e977bd058b1c-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.610432 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.610407    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz2qw\" (UniqueName: \"kubernetes.io/projected/a373b999-a0cb-4016-9415-e977bd058b1c-kube-api-access-fz2qw\") pod \"maas-default-gateway-openshift-default-66db998887-xttc4\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.888591 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.888559    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:09.966614 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.966407    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"]
Oct 30 18:50:09.969013 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:50:09.968984    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76921ad3_c8c7_4de8_b9ef_5198a8c94327.slice/crio-70f19cc2875d9b1b4763925f4ca247fccc39d36a42caf35c100239d5351abf9f WatchSource:0}: Error finding container 70f19cc2875d9b1b4763925f4ca247fccc39d36a42caf35c100239d5351abf9f: Status 404 returned error can't find the container with id 70f19cc2875d9b1b4763925f4ca247fccc39d36a42caf35c100239d5351abf9f
Oct 30 18:50:09.971591 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.971559    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:50:09.971680 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.971622    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:50:09.971680 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:09.971653    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:50:10.271788 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:10.271759    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"]
Oct 30 18:50:10.274074 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:50:10.274047    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda373b999_a0cb_4016_9415_e977bd058b1c.slice/crio-38c7dd829e8e9faad5f7ccb2b3881f3beef66618f4a3ed997d036173a95bc8ef WatchSource:0}: Error finding container 38c7dd829e8e9faad5f7ccb2b3881f3beef66618f4a3ed997d036173a95bc8ef: Status 404 returned error can't find the container with id 38c7dd829e8e9faad5f7ccb2b3881f3beef66618f4a3ed997d036173a95bc8ef
Oct 30 18:50:10.277049 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:10.277015    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:50:10.277150 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:10.277097    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:50:10.277217 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:10.277151    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:50:10.401484 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:10.401440    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf" event={"ID":"76921ad3-c8c7-4de8-b9ef-5198a8c94327","Type":"ContainerStarted","Data":"7ac5e6072471f1654d6ffa6bf7ed1abaa8f3818b44ae7078bbcb1361c390ee11"}
Oct 30 18:50:10.401681 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:10.401550    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf" event={"ID":"76921ad3-c8c7-4de8-b9ef-5198a8c94327","Type":"ContainerStarted","Data":"70f19cc2875d9b1b4763925f4ca247fccc39d36a42caf35c100239d5351abf9f"}
Oct 30 18:50:10.403331 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:10.403304    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4" event={"ID":"a373b999-a0cb-4016-9415-e977bd058b1c","Type":"ContainerStarted","Data":"024021fc61abae1ad6a00217f2b16712f2abc1a3441d71f547761fa271efaa4a"}
Oct 30 18:50:10.403468 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:10.403344    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4" event={"ID":"a373b999-a0cb-4016-9415-e977bd058b1c","Type":"ContainerStarted","Data":"38c7dd829e8e9faad5f7ccb2b3881f3beef66618f4a3ed997d036173a95bc8ef"}
Oct 30 18:50:10.527740 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:10.524484    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf" podStartSLOduration=1.524462909 podStartE2EDuration="1.524462909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:10.521886542 +0000 UTC m=+931.233635241" watchObservedRunningTime="2026-10-30 18:50:10.524462909 +0000 UTC m=+931.236211607"
Oct 30 18:50:10.546975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:10.537069    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:10.893350 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:10.890161    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:11.550974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:11.550089    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-qxlbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 18:50:11.550974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:11.550173    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf" podUID="76921ad3-c8c7-4de8-b9ef-5198a8c94327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 18:50:11.891089 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:11.890776    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-xttc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 18:50:11.891089 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:11.890833    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4" podUID="a373b999-a0cb-4016-9415-e977bd058b1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 18:50:12.534162 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:12.534129    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-qxlbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 18:50:12.534445 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:12.534419    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf" podUID="76921ad3-c8c7-4de8-b9ef-5198a8c94327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 18:50:12.889993 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:12.889556    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-xttc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 18:50:12.889993 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:12.889621    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4" podUID="a373b999-a0cb-4016-9415-e977bd058b1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 18:50:13.533663 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:13.533632    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-qxlbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 18:50:13.533841 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:13.533691    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf" podUID="76921ad3-c8c7-4de8-b9ef-5198a8c94327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 18:50:13.893567 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:13.893532    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-xttc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 18:50:13.894230 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:13.894176    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4" podUID="a373b999-a0cb-4016-9415-e977bd058b1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 18:50:13.899564 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:13.899534    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:14.533802 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:14.533769    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-qxlbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 18:50:14.543028 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:14.533816    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf" podUID="76921ad3-c8c7-4de8-b9ef-5198a8c94327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 18:50:14.890431 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:14.890380    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-xttc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 18:50:14.890646 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:14.890443    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4" podUID="a373b999-a0cb-4016-9415-e977bd058b1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 18:50:15.543212 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:15.537367    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-qxlbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 18:50:15.543212 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:15.537432    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf" podUID="76921ad3-c8c7-4de8-b9ef-5198a8c94327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 18:50:15.889043 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:15.889010    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-xttc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 18:50:15.889247 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:15.889181    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4" podUID="a373b999-a0cb-4016-9415-e977bd058b1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 18:50:16.534424 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:16.534386    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-qxlbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 18:50:16.534732 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:16.534702    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf" podUID="76921ad3-c8c7-4de8-b9ef-5198a8c94327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 18:50:16.895296 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:16.895264    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-xttc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 18:50:16.896533 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:16.896498    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4" podUID="a373b999-a0cb-4016-9415-e977bd058b1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 18:50:17.033998 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:17.032251    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:17.535012 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:17.534936    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-qxlbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 18:50:17.535335 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:17.535293    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf" podUID="76921ad3-c8c7-4de8-b9ef-5198a8c94327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 18:50:17.889543 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:17.889082    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-xttc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 18:50:17.889543 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:17.889140    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4" podUID="a373b999-a0cb-4016-9415-e977bd058b1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 18:50:19.148977 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:19.148387    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:19.574156 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:19.574119    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-qxlbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:50:19.574508 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:19.574471    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf" podUID="76921ad3-c8c7-4de8-b9ef-5198a8c94327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:50:19.933178 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:19.933086    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-xttc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:50:19.933178 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:19.933147    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4" podUID="a373b999-a0cb-4016-9415-e977bd058b1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:50:20.026566 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:20.026530    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:20.026841 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:20.026815    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:20.028345 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:20.028317    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 18:50:20.164146 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:20.164070    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4" podStartSLOduration=11.164050006 podStartE2EDuration="11.164050006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:10.649347068 +0000 UTC m=+931.361095792" watchObservedRunningTime="2026-10-30 18:50:20.164050006 +0000 UTC m=+940.875798704"
Oct 30 18:50:20.586485 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:20.586447    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-qxlbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:50:20.586812 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:20.586781    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf" podUID="76921ad3-c8c7-4de8-b9ef-5198a8c94327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:50:21.629327 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:21.629280    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-qxlbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:50:21.629943 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:21.629909    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf" podUID="76921ad3-c8c7-4de8-b9ef-5198a8c94327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:50:21.984478 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:21.984382    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:21.986225 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:21.986191    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:21.988404 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:21.988377    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 18:50:22.744827 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:22.744794    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"]
Oct 30 18:50:22.748720 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:22.748699    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"
Oct 30 18:50:22.801199 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:22.801160    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Oct 30 18:50:22.801454 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:22.801413    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Oct 30 18:50:22.801454 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:22.801418    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Oct 30 18:50:22.801454 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:22.801453    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Oct 30 18:50:22.801642 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:22.801493    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-sqhtd\""
Oct 30 18:50:22.825480 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:22.825451    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f6451130-0a4d-489f-a835-89512414c68a-routing-overlay-volume\") pod \"payload-processing-external-model-64d7c87b6-l4kpk\" (UID: \"f6451130-0a4d-489f-a835-89512414c68a\") " pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"
Oct 30 18:50:22.825653 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:22.825503    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f6451130-0a4d-489f-a835-89512414c68a-provider-credentials\") pod \"payload-processing-external-model-64d7c87b6-l4kpk\" (UID: \"f6451130-0a4d-489f-a835-89512414c68a\") " pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"
Oct 30 18:50:22.825653 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:22.825612    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6451130-0a4d-489f-a835-89512414c68a-plugins-config-volume\") pod \"payload-processing-external-model-64d7c87b6-l4kpk\" (UID: \"f6451130-0a4d-489f-a835-89512414c68a\") " pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"
Oct 30 18:50:22.825795 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:22.825761    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7m4k\" (UniqueName: \"kubernetes.io/projected/f6451130-0a4d-489f-a835-89512414c68a-kube-api-access-q7m4k\") pod \"payload-processing-external-model-64d7c87b6-l4kpk\" (UID: \"f6451130-0a4d-489f-a835-89512414c68a\") " pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"
Oct 30 18:50:22.850523 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:22.850491    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-0c229ede8440\""
Oct 30 18:50:22.852827 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:22.852803    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"]
Oct 30 18:50:22.927232 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:22.927196    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f6451130-0a4d-489f-a835-89512414c68a-routing-overlay-volume\") pod \"payload-processing-external-model-64d7c87b6-l4kpk\" (UID: \"f6451130-0a4d-489f-a835-89512414c68a\") " pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"
Oct 30 18:50:22.927459 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:22.927246    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f6451130-0a4d-489f-a835-89512414c68a-provider-credentials\") pod \"payload-processing-external-model-64d7c87b6-l4kpk\" (UID: \"f6451130-0a4d-489f-a835-89512414c68a\") " pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"
Oct 30 18:50:22.927459 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:22.927315    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6451130-0a4d-489f-a835-89512414c68a-plugins-config-volume\") pod \"payload-processing-external-model-64d7c87b6-l4kpk\" (UID: \"f6451130-0a4d-489f-a835-89512414c68a\") " pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"
Oct 30 18:50:22.927459 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:22.927423    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7m4k\" (UniqueName: \"kubernetes.io/projected/f6451130-0a4d-489f-a835-89512414c68a-kube-api-access-q7m4k\") pod \"payload-processing-external-model-64d7c87b6-l4kpk\" (UID: \"f6451130-0a4d-489f-a835-89512414c68a\") " pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"
Oct 30 18:50:22.928042 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:22.928016    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f6451130-0a4d-489f-a835-89512414c68a-routing-overlay-volume\") pod \"payload-processing-external-model-64d7c87b6-l4kpk\" (UID: \"f6451130-0a4d-489f-a835-89512414c68a\") " pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"
Oct 30 18:50:22.928136 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:22.928122    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6451130-0a4d-489f-a835-89512414c68a-plugins-config-volume\") pod \"payload-processing-external-model-64d7c87b6-l4kpk\" (UID: \"f6451130-0a4d-489f-a835-89512414c68a\") " pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"
Oct 30 18:50:22.929901 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:22.929881    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f6451130-0a4d-489f-a835-89512414c68a-provider-credentials\") pod \"payload-processing-external-model-64d7c87b6-l4kpk\" (UID: \"f6451130-0a4d-489f-a835-89512414c68a\") " pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"
Oct 30 18:50:22.989093 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:22.989061    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7m4k\" (UniqueName: \"kubernetes.io/projected/f6451130-0a4d-489f-a835-89512414c68a-kube-api-access-q7m4k\") pod \"payload-processing-external-model-64d7c87b6-l4kpk\" (UID: \"f6451130-0a4d-489f-a835-89512414c68a\") " pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"
Oct 30 18:50:23.064139 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:23.064110    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"
Oct 30 18:50:23.441041 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:23.441018    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"]
Oct 30 18:50:23.444089 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:50:23.444064    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6451130_0a4d_489f_a835_89512414c68a.slice/crio-00bd88f45881ad9e03a8480098197895e199c49f95eb65545490982f36f573c8 WatchSource:0}: Error finding container 00bd88f45881ad9e03a8480098197895e199c49f95eb65545490982f36f573c8: Status 404 returned error can't find the container with id 00bd88f45881ad9e03a8480098197895e199c49f95eb65545490982f36f573c8
Oct 30 18:50:23.495582 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:23.495557    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk" event={"ID":"f6451130-0a4d-489f-a835-89512414c68a","Type":"ContainerStarted","Data":"00bd88f45881ad9e03a8480098197895e199c49f95eb65545490982f36f573c8"}
Oct 30 18:50:23.932796 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:23.932751    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:24.502475 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:24.502437    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk" event={"ID":"f6451130-0a4d-489f-a835-89512414c68a","Type":"ContainerStarted","Data":"e95900eb222cedab8e4461637866858f7f6467cd004724b185d43a8e7a3dada7"}
Oct 30 18:50:24.502653 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:24.502573    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"
Oct 30 18:50:24.619736 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:24.619684    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk" podStartSLOduration=2.619669355 podStartE2EDuration="2.619669355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:24.618000113 +0000 UTC m=+945.329748809" watchObservedRunningTime="2026-10-30 18:50:24.619669355 +0000 UTC m=+945.331418051"
Oct 30 18:50:28.956985 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:28.954282    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:33.848214 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:33.848122    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:35.510758 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:35.510717    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk" podUID="f6451130-0a4d-489f-a835-89512414c68a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 18:50:38.203405 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:50:38.203364    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:50:38.760197 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:38.760057    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:43.635268 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:43.635229    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:45.509368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:45.509334    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk" podUID="f6451130-0a4d-489f-a835-89512414c68a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 18:50:48.475415 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:48.475377    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:50.210576 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:50.210527    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"]
Oct 30 18:50:50.211904 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:50.211878    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk" podUID="f6451130-0a4d-489f-a835-89512414c68a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 18:50:50.249204 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:50.249168    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"]
Oct 30 18:50:50.254486 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:50.254465    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"
Oct 30 18:50:50.310035 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:50.310007    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-01eb9baaf090\""
Oct 30 18:50:50.327264 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:50.327233    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"]
Oct 30 18:50:50.411305 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:50.411275    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/fd5b9496-731a-4bce-97a0-43d1cef93a28-provider-credentials\") pod \"payload-processing-external-model-59c7d6dc54-rrwrt\" (UID: \"fd5b9496-731a-4bce-97a0-43d1cef93a28\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"
Oct 30 18:50:50.411305 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:50.411308    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szfp7\" (UniqueName: \"kubernetes.io/projected/fd5b9496-731a-4bce-97a0-43d1cef93a28-kube-api-access-szfp7\") pod \"payload-processing-external-model-59c7d6dc54-rrwrt\" (UID: \"fd5b9496-731a-4bce-97a0-43d1cef93a28\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"
Oct 30 18:50:50.411518 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:50.411472    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/fd5b9496-731a-4bce-97a0-43d1cef93a28-routing-overlay-volume\") pod \"payload-processing-external-model-59c7d6dc54-rrwrt\" (UID: \"fd5b9496-731a-4bce-97a0-43d1cef93a28\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"
Oct 30 18:50:50.411518 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:50.411509    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd5b9496-731a-4bce-97a0-43d1cef93a28-plugins-config-volume\") pod \"payload-processing-external-model-59c7d6dc54-rrwrt\" (UID: \"fd5b9496-731a-4bce-97a0-43d1cef93a28\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"
Oct 30 18:50:50.512519 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:50.512439    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/fd5b9496-731a-4bce-97a0-43d1cef93a28-routing-overlay-volume\") pod \"payload-processing-external-model-59c7d6dc54-rrwrt\" (UID: \"fd5b9496-731a-4bce-97a0-43d1cef93a28\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"
Oct 30 18:50:50.512519 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:50.512487    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd5b9496-731a-4bce-97a0-43d1cef93a28-plugins-config-volume\") pod \"payload-processing-external-model-59c7d6dc54-rrwrt\" (UID: \"fd5b9496-731a-4bce-97a0-43d1cef93a28\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"
Oct 30 18:50:50.512703 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:50.512554    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/fd5b9496-731a-4bce-97a0-43d1cef93a28-provider-credentials\") pod \"payload-processing-external-model-59c7d6dc54-rrwrt\" (UID: \"fd5b9496-731a-4bce-97a0-43d1cef93a28\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"
Oct 30 18:50:50.512703 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:50.512570    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szfp7\" (UniqueName: \"kubernetes.io/projected/fd5b9496-731a-4bce-97a0-43d1cef93a28-kube-api-access-szfp7\") pod \"payload-processing-external-model-59c7d6dc54-rrwrt\" (UID: \"fd5b9496-731a-4bce-97a0-43d1cef93a28\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"
Oct 30 18:50:50.513311 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:50.513279    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/fd5b9496-731a-4bce-97a0-43d1cef93a28-routing-overlay-volume\") pod \"payload-processing-external-model-59c7d6dc54-rrwrt\" (UID: \"fd5b9496-731a-4bce-97a0-43d1cef93a28\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"
Oct 30 18:50:50.513435 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:50.513351    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd5b9496-731a-4bce-97a0-43d1cef93a28-plugins-config-volume\") pod \"payload-processing-external-model-59c7d6dc54-rrwrt\" (UID: \"fd5b9496-731a-4bce-97a0-43d1cef93a28\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"
Oct 30 18:50:50.515037 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:50.515018    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/fd5b9496-731a-4bce-97a0-43d1cef93a28-provider-credentials\") pod \"payload-processing-external-model-59c7d6dc54-rrwrt\" (UID: \"fd5b9496-731a-4bce-97a0-43d1cef93a28\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"
Oct 30 18:50:50.573643 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:50.573612    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szfp7\" (UniqueName: \"kubernetes.io/projected/fd5b9496-731a-4bce-97a0-43d1cef93a28-kube-api-access-szfp7\") pod \"payload-processing-external-model-59c7d6dc54-rrwrt\" (UID: \"fd5b9496-731a-4bce-97a0-43d1cef93a28\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"
Oct 30 18:50:50.648753 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:50.648718    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:50.867750 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:50.867715    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"
Oct 30 18:50:51.245722 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:51.245692    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"]
Oct 30 18:50:51.248079 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:50:51.248053    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd5b9496_731a_4bce_97a0_43d1cef93a28.slice/crio-1c2f2cefdf831c35fab0997c22da901746add27ef234a34a484258387a329c4b WatchSource:0}: Error finding container 1c2f2cefdf831c35fab0997c22da901746add27ef234a34a484258387a329c4b: Status 404 returned error can't find the container with id 1c2f2cefdf831c35fab0997c22da901746add27ef234a34a484258387a329c4b
Oct 30 18:50:51.650937 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:51.650903    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:51.696694 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:51.696661    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt" event={"ID":"fd5b9496-731a-4bce-97a0-43d1cef93a28","Type":"ContainerStarted","Data":"b51361d32f37dae7bf16af5760f28b3a05a6bf470dc013890ac0f0e6106cecab"}
Oct 30 18:50:51.696854 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:51.696717    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt" event={"ID":"fd5b9496-731a-4bce-97a0-43d1cef93a28","Type":"ContainerStarted","Data":"1c2f2cefdf831c35fab0997c22da901746add27ef234a34a484258387a329c4b"}
Oct 30 18:50:51.696854 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:51.696825    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"
Oct 30 18:50:51.814899 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:51.814849    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt" podStartSLOduration=1.814835987 podStartE2EDuration="1.814835987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:51.81325737 +0000 UTC m=+972.525006068" watchObservedRunningTime="2026-10-30 18:50:51.814835987 +0000 UTC m=+972.526584684"
Oct 30 18:50:53.767974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:53.763800    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:55.211386 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:55.211333    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk" podUID="f6451130-0a4d-489f-a835-89512414c68a" containerName="payload-processing" containerID="cri-o://e95900eb222cedab8e4461637866858f7f6467cd004724b185d43a8e7a3dada7" gracePeriod=25
Oct 30 18:50:56.196743 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:56.196703    5820 generic.go:356] "Generic (PLEG): container finished" podID="f6451130-0a4d-489f-a835-89512414c68a" containerID="e95900eb222cedab8e4461637866858f7f6467cd004724b185d43a8e7a3dada7" exitCode=0
Oct 30 18:50:56.203977 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:56.197138    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk" event={"ID":"f6451130-0a4d-489f-a835-89512414c68a","Type":"ContainerDied","Data":"e95900eb222cedab8e4461637866858f7f6467cd004724b185d43a8e7a3dada7"}
Oct 30 18:50:56.276158 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:56.276125    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"
Oct 30 18:50:56.393974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:56.393640    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6451130-0a4d-489f-a835-89512414c68a-kube-api-access-q7m4k\" (UniqueName: \"kubernetes.io/projected/f6451130-0a4d-489f-a835-89512414c68a-kube-api-access-q7m4k\") pod \"f6451130-0a4d-489f-a835-89512414c68a\" (UID: \"f6451130-0a4d-489f-a835-89512414c68a\") "
Oct 30 18:50:56.393974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:56.393743    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6451130-0a4d-489f-a835-89512414c68a-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f6451130-0a4d-489f-a835-89512414c68a-routing-overlay-volume\") pod \"f6451130-0a4d-489f-a835-89512414c68a\" (UID: \"f6451130-0a4d-489f-a835-89512414c68a\") "
Oct 30 18:50:56.393974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:56.393795    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6451130-0a4d-489f-a835-89512414c68a-provider-credentials\" (UniqueName: \"kubernetes.io/projected/f6451130-0a4d-489f-a835-89512414c68a-provider-credentials\") pod \"f6451130-0a4d-489f-a835-89512414c68a\" (UID: \"f6451130-0a4d-489f-a835-89512414c68a\") "
Oct 30 18:50:56.393974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:56.393886    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6451130-0a4d-489f-a835-89512414c68a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6451130-0a4d-489f-a835-89512414c68a-plugins-config-volume\") pod \"f6451130-0a4d-489f-a835-89512414c68a\" (UID: \"f6451130-0a4d-489f-a835-89512414c68a\") "
Oct 30 18:50:56.398010 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:56.394705    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6451130-0a4d-489f-a835-89512414c68a-routing-overlay-volume" pod "f6451130-0a4d-489f-a835-89512414c68a" (UID: "f6451130-0a4d-489f-a835-89512414c68a"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:50:56.398010 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:56.395213    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6451130-0a4d-489f-a835-89512414c68a-plugins-config-volume" pod "f6451130-0a4d-489f-a835-89512414c68a" (UID: "f6451130-0a4d-489f-a835-89512414c68a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:50:56.417262 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:56.417223    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6451130-0a4d-489f-a835-89512414c68a-kube-api-access-q7m4k" pod "f6451130-0a4d-489f-a835-89512414c68a" (UID: "f6451130-0a4d-489f-a835-89512414c68a"). InnerVolumeSpecName "kube-api-access-q7m4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:50:56.594673 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:56.594630    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7m4k\" (UniqueName: \"kubernetes.io/projected/f6451130-0a4d-489f-a835-89512414c68a-kube-api-access-q7m4k\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:50:56.594919 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:56.594901    5820 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f6451130-0a4d-489f-a835-89512414c68a-routing-overlay-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:50:56.606160 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:56.606135    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6451130-0a4d-489f-a835-89512414c68a-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:50:56.611231 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:56.611200    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6451130-0a4d-489f-a835-89512414c68a-provider-credentials" pod "f6451130-0a4d-489f-a835-89512414c68a" (UID: "f6451130-0a4d-489f-a835-89512414c68a"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:50:56.711326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:56.711234    5820 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f6451130-0a4d-489f-a835-89512414c68a-provider-credentials\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:50:57.212317 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:57.212278    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk" event={"ID":"f6451130-0a4d-489f-a835-89512414c68a","Type":"ContainerDied","Data":"00bd88f45881ad9e03a8480098197895e199c49f95eb65545490982f36f573c8"}
Oct 30 18:50:57.212627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:57.212607    5820 scope.go:122] "RemoveContainer" containerID="e95900eb222cedab8e4461637866858f7f6467cd004724b185d43a8e7a3dada7"
Oct 30 18:50:57.212906 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:57.212887    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"
Oct 30 18:50:57.390574 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:57.390536    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"]
Oct 30 18:50:57.438839 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:57.438811    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-64d7c87b6-l4kpk"]
Oct 30 18:50:58.850804 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:58.850769    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:59.437005 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:50:59.436973    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6451130-0a4d-489f-a835-89512414c68a" path="/var/lib/kubelet/pods/f6451130-0a4d-489f-a835-89512414c68a/volumes"
Oct 30 18:51:01.751339 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:01.751304    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:02.706064 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:02.706020    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt" podUID="fd5b9496-731a-4bce-97a0-43d1cef93a28" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 18:51:03.657554 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:03.657518    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:05.530449 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:05.530413    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:05.707584 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:05.707549    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:05.914859 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:05.914826    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:06.091358 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:06.091326    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:08.203560 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:51:08.203527    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:51:08.424795 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:08.424758    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:10.206216 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.206181    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"]
Oct 30 18:51:10.207557 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.207529    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt" podUID="fd5b9496-731a-4bce-97a0-43d1cef93a28" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 18:51:10.240696 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.240666    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-896496998-wbhwr"]
Oct 30 18:51:10.241217 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.241204    5820 cpu_manager.go:414] "containerMap: removing container" podUID="f6451130-0a4d-489f-a835-89512414c68a" containerName="payload-processing"
Oct 30 18:51:10.241276 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.241219    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6451130-0a4d-489f-a835-89512414c68a" containerName="payload-processing"
Oct 30 18:51:10.241313 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.241293    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6451130-0a4d-489f-a835-89512414c68a" containerName="payload-processing"
Oct 30 18:51:10.244736 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.244720    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr"
Oct 30 18:51:10.297380 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.297349    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-c9dc70e8fa4c\""
Oct 30 18:51:10.297380 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.297348    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-588f9d9488d3\""
Oct 30 18:51:10.319691 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.319645    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87a24e2c-6c3e-4203-9030-e009165d27a5-plugins-config-volume\") pod \"payload-processing-external-model-896496998-wbhwr\" (UID: \"87a24e2c-6c3e-4203-9030-e009165d27a5\") " pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr"
Oct 30 18:51:10.319882 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.319840    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/87a24e2c-6c3e-4203-9030-e009165d27a5-provider-credentials\") pod \"payload-processing-external-model-896496998-wbhwr\" (UID: \"87a24e2c-6c3e-4203-9030-e009165d27a5\") " pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr"
Oct 30 18:51:10.320077 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.320021    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/87a24e2c-6c3e-4203-9030-e009165d27a5-routing-overlay-volume\") pod \"payload-processing-external-model-896496998-wbhwr\" (UID: \"87a24e2c-6c3e-4203-9030-e009165d27a5\") " pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr"
Oct 30 18:51:10.320259 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.320233    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zmnl\" (UniqueName: \"kubernetes.io/projected/87a24e2c-6c3e-4203-9030-e009165d27a5-kube-api-access-2zmnl\") pod \"payload-processing-external-model-896496998-wbhwr\" (UID: \"87a24e2c-6c3e-4203-9030-e009165d27a5\") " pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr"
Oct 30 18:51:10.321685 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.321666    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-896496998-wbhwr"]
Oct 30 18:51:10.421826 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.421791    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zmnl\" (UniqueName: \"kubernetes.io/projected/87a24e2c-6c3e-4203-9030-e009165d27a5-kube-api-access-2zmnl\") pod \"payload-processing-external-model-896496998-wbhwr\" (UID: \"87a24e2c-6c3e-4203-9030-e009165d27a5\") " pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr"
Oct 30 18:51:10.422070 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.421849    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87a24e2c-6c3e-4203-9030-e009165d27a5-plugins-config-volume\") pod \"payload-processing-external-model-896496998-wbhwr\" (UID: \"87a24e2c-6c3e-4203-9030-e009165d27a5\") " pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr"
Oct 30 18:51:10.422070 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.421927    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/87a24e2c-6c3e-4203-9030-e009165d27a5-provider-credentials\") pod \"payload-processing-external-model-896496998-wbhwr\" (UID: \"87a24e2c-6c3e-4203-9030-e009165d27a5\") " pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr"
Oct 30 18:51:10.422070 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.422019    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/87a24e2c-6c3e-4203-9030-e009165d27a5-routing-overlay-volume\") pod \"payload-processing-external-model-896496998-wbhwr\" (UID: \"87a24e2c-6c3e-4203-9030-e009165d27a5\") " pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr"
Oct 30 18:51:10.422629 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.422605    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/87a24e2c-6c3e-4203-9030-e009165d27a5-routing-overlay-volume\") pod \"payload-processing-external-model-896496998-wbhwr\" (UID: \"87a24e2c-6c3e-4203-9030-e009165d27a5\") " pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr"
Oct 30 18:51:10.422718 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.422654    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87a24e2c-6c3e-4203-9030-e009165d27a5-plugins-config-volume\") pod \"payload-processing-external-model-896496998-wbhwr\" (UID: \"87a24e2c-6c3e-4203-9030-e009165d27a5\") " pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr"
Oct 30 18:51:10.424596 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.424577    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/87a24e2c-6c3e-4203-9030-e009165d27a5-provider-credentials\") pod \"payload-processing-external-model-896496998-wbhwr\" (UID: \"87a24e2c-6c3e-4203-9030-e009165d27a5\") " pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr"
Oct 30 18:51:10.483370 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.483301    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zmnl\" (UniqueName: \"kubernetes.io/projected/87a24e2c-6c3e-4203-9030-e009165d27a5-kube-api-access-2zmnl\") pod \"payload-processing-external-model-896496998-wbhwr\" (UID: \"87a24e2c-6c3e-4203-9030-e009165d27a5\") " pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr"
Oct 30 18:51:10.558600 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.558567    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr"
Oct 30 18:51:10.934130 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:10.934107    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-896496998-wbhwr"]
Oct 30 18:51:10.937101 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:51:10.937072    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87a24e2c_6c3e_4203_9030_e009165d27a5.slice/crio-893fdca585c6180ed5044d0cfecf98813cefa44245d5aed472761d7e3e53b1ec WatchSource:0}: Error finding container 893fdca585c6180ed5044d0cfecf98813cefa44245d5aed472761d7e3e53b1ec: Status 404 returned error can't find the container with id 893fdca585c6180ed5044d0cfecf98813cefa44245d5aed472761d7e3e53b1ec
Oct 30 18:51:11.294067 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:11.293941    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr" event={"ID":"87a24e2c-6c3e-4203-9030-e009165d27a5","Type":"ContainerStarted","Data":"c97d335ce9163374c5cdc6e32ca01804f763fb01656f0fa381205ffa1d11dbbc"}
Oct 30 18:51:11.294067 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:11.294034    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr" event={"ID":"87a24e2c-6c3e-4203-9030-e009165d27a5","Type":"ContainerStarted","Data":"893fdca585c6180ed5044d0cfecf98813cefa44245d5aed472761d7e3e53b1ec"}
Oct 30 18:51:11.294588 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:11.294161    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr"
Oct 30 18:51:11.412449 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:11.411928    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr" podStartSLOduration=1.411909783 podStartE2EDuration="1.411909783s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:51:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:51:11.411365148 +0000 UTC m=+992.123113846" watchObservedRunningTime="2026-10-30 18:51:11.411909783 +0000 UTC m=+992.123658482"
Oct 30 18:51:13.455906 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:13.455869    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:15.214975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:15.214040    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt" podUID="fd5b9496-731a-4bce-97a0-43d1cef93a28" containerName="payload-processing" containerID="cri-o://b51361d32f37dae7bf16af5760f28b3a05a6bf470dc013890ac0f0e6106cecab" gracePeriod=25
Oct 30 18:51:15.350285 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:15.350194    5820 generic.go:356] "Generic (PLEG): container finished" podID="fd5b9496-731a-4bce-97a0-43d1cef93a28" containerID="b51361d32f37dae7bf16af5760f28b3a05a6bf470dc013890ac0f0e6106cecab" exitCode=0
Oct 30 18:51:15.350417 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:15.350287    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt" event={"ID":"fd5b9496-731a-4bce-97a0-43d1cef93a28","Type":"ContainerDied","Data":"b51361d32f37dae7bf16af5760f28b3a05a6bf470dc013890ac0f0e6106cecab"}
Oct 30 18:51:15.488680 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:15.488653    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"
Oct 30 18:51:15.535860 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:15.535830    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd5b9496-731a-4bce-97a0-43d1cef93a28-kube-api-access-szfp7\" (UniqueName: \"kubernetes.io/projected/fd5b9496-731a-4bce-97a0-43d1cef93a28-kube-api-access-szfp7\") pod \"fd5b9496-731a-4bce-97a0-43d1cef93a28\" (UID: \"fd5b9496-731a-4bce-97a0-43d1cef93a28\") "
Oct 30 18:51:15.536051 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:15.535916    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd5b9496-731a-4bce-97a0-43d1cef93a28-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/fd5b9496-731a-4bce-97a0-43d1cef93a28-routing-overlay-volume\") pod \"fd5b9496-731a-4bce-97a0-43d1cef93a28\" (UID: \"fd5b9496-731a-4bce-97a0-43d1cef93a28\") "
Oct 30 18:51:15.536051 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:15.535982    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd5b9496-731a-4bce-97a0-43d1cef93a28-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd5b9496-731a-4bce-97a0-43d1cef93a28-plugins-config-volume\") pod \"fd5b9496-731a-4bce-97a0-43d1cef93a28\" (UID: \"fd5b9496-731a-4bce-97a0-43d1cef93a28\") "
Oct 30 18:51:15.536169 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:15.536108    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd5b9496-731a-4bce-97a0-43d1cef93a28-provider-credentials\" (UniqueName: \"kubernetes.io/projected/fd5b9496-731a-4bce-97a0-43d1cef93a28-provider-credentials\") pod \"fd5b9496-731a-4bce-97a0-43d1cef93a28\" (UID: \"fd5b9496-731a-4bce-97a0-43d1cef93a28\") "
Oct 30 18:51:15.536303 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:15.536278    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd5b9496-731a-4bce-97a0-43d1cef93a28-routing-overlay-volume" pod "fd5b9496-731a-4bce-97a0-43d1cef93a28" (UID: "fd5b9496-731a-4bce-97a0-43d1cef93a28"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:51:15.536408 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:15.536383    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd5b9496-731a-4bce-97a0-43d1cef93a28-plugins-config-volume" pod "fd5b9496-731a-4bce-97a0-43d1cef93a28" (UID: "fd5b9496-731a-4bce-97a0-43d1cef93a28"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:51:15.536904 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:15.536872    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd5b9496-731a-4bce-97a0-43d1cef93a28-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:51:15.537022 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:15.536913    5820 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/fd5b9496-731a-4bce-97a0-43d1cef93a28-routing-overlay-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:51:15.538293 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:15.538265    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd5b9496-731a-4bce-97a0-43d1cef93a28-kube-api-access-szfp7" pod "fd5b9496-731a-4bce-97a0-43d1cef93a28" (UID: "fd5b9496-731a-4bce-97a0-43d1cef93a28"). InnerVolumeSpecName "kube-api-access-szfp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:51:15.538386 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:15.538346    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd5b9496-731a-4bce-97a0-43d1cef93a28-provider-credentials" pod "fd5b9496-731a-4bce-97a0-43d1cef93a28" (UID: "fd5b9496-731a-4bce-97a0-43d1cef93a28"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:51:15.638326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:15.638286    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szfp7\" (UniqueName: \"kubernetes.io/projected/fd5b9496-731a-4bce-97a0-43d1cef93a28-kube-api-access-szfp7\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:51:15.638326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:15.638317    5820 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/fd5b9496-731a-4bce-97a0-43d1cef93a28-provider-credentials\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:51:16.357001 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:16.356944    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt" event={"ID":"fd5b9496-731a-4bce-97a0-43d1cef93a28","Type":"ContainerDied","Data":"1c2f2cefdf831c35fab0997c22da901746add27ef234a34a484258387a329c4b"}
Oct 30 18:51:16.357001 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:16.356990    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"
Oct 30 18:51:16.357493 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:16.357020    5820 scope.go:122] "RemoveContainer" containerID="b51361d32f37dae7bf16af5760f28b3a05a6bf470dc013890ac0f0e6106cecab"
Oct 30 18:51:16.520658 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:16.520621    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"]
Oct 30 18:51:16.527487 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:16.527464    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-59c7d6dc54-rrwrt"]
Oct 30 18:51:17.436521 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:17.436451    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd5b9496-731a-4bce-97a0-43d1cef93a28" path="/var/lib/kubelet/pods/fd5b9496-731a-4bce-97a0-43d1cef93a28/volumes"
Oct 30 18:51:18.467973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:18.448540    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:22.306968 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:22.306927    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr"
Oct 30 18:51:23.368116 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:23.360770    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:28.241409 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:28.241362    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:33.633082 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:33.633039    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:38.204088 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:51:38.204047    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:51:38.470197 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:38.470109    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:42.425155 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.425116    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"]
Oct 30 18:51:42.425660 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.425648    5820 cpu_manager.go:414] "containerMap: removing container" podUID="fd5b9496-731a-4bce-97a0-43d1cef93a28" containerName="payload-processing"
Oct 30 18:51:42.425704 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.425662    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd5b9496-731a-4bce-97a0-43d1cef93a28" containerName="payload-processing"
Oct 30 18:51:42.425755 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.425746    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd5b9496-731a-4bce-97a0-43d1cef93a28" containerName="payload-processing"
Oct 30 18:51:42.430161 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.430143    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.482544 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.482511    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-fd61e3-openshift-default-dockercfg-tn6nh\""
Oct 30 18:51:42.529486 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.529171    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.529486 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.529253    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7drp\" (UniqueName: \"kubernetes.io/projected/72548522-bba4-4f29-92ea-4d41d72e88b1-kube-api-access-j7drp\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.529486 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.529318    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.529486 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.529371    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.529486 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.529491    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.529880 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.529516    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.529880 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.529560    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.529880 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.529593    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.529880 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.529615    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72548522-bba4-4f29-92ea-4d41d72e88b1-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.531639 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.531614    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"]
Oct 30 18:51:42.630434 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.630397    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.630653 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.630440    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.630653 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.630479    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.630653 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.630509    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.630653 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.630532    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72548522-bba4-4f29-92ea-4d41d72e88b1-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.630890 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.630676    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.630890 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.630762    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7drp\" (UniqueName: \"kubernetes.io/projected/72548522-bba4-4f29-92ea-4d41d72e88b1-kube-api-access-j7drp\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.630890 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.630831    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.630890 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.630883    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.631167 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.630885    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.631167 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.630978    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.631167 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.631057    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.631167 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.631065    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.631380 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.631318    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72548522-bba4-4f29-92ea-4d41d72e88b1-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.633251 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.633224    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.633357 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.633344    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.692645 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.692571    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.692780 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.692710    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7drp\" (UniqueName: \"kubernetes.io/projected/72548522-bba4-4f29-92ea-4d41d72e88b1-kube-api-access-j7drp\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:42.744060 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:42.744021    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:43.175184 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:43.175146    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"]
Oct 30 18:51:43.180223 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:51:43.180195    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72548522_bba4_4f29_92ea_4d41d72e88b1.slice/crio-cf4c6fcc438037db85d7f7d66a37af5f0aed09fdae2d4c41ac4ce11d070314b9 WatchSource:0}: Error finding container cf4c6fcc438037db85d7f7d66a37af5f0aed09fdae2d4c41ac4ce11d070314b9: Status 404 returned error can't find the container with id cf4c6fcc438037db85d7f7d66a37af5f0aed09fdae2d4c41ac4ce11d070314b9
Oct 30 18:51:43.182519 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:43.182484    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:51:43.182623 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:43.182550    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:51:43.182623 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:43.182581    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:51:43.335341 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:43.335300    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:43.553134 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:43.553102    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs" event={"ID":"72548522-bba4-4f29-92ea-4d41d72e88b1","Type":"ContainerStarted","Data":"89dd0215184eaf3a324caddfa14a4745bfaa69895726f93d93c6d823e6269eea"}
Oct 30 18:51:43.553619 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:43.553157    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs" event={"ID":"72548522-bba4-4f29-92ea-4d41d72e88b1","Type":"ContainerStarted","Data":"cf4c6fcc438037db85d7f7d66a37af5f0aed09fdae2d4c41ac4ce11d070314b9"}
Oct 30 18:51:43.674622 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:43.674567    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs" podStartSLOduration=1.674552128 podStartE2EDuration="1.674552128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:51:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:51:43.672215463 +0000 UTC m=+1024.383964164" watchObservedRunningTime="2026-10-30 18:51:43.674552128 +0000 UTC m=+1024.386300825"
Oct 30 18:51:43.744927 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:43.744893    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:44.659845 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:44.659797    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:44.755099 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:44.755061    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 18:51:44.755375 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:44.755347    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs" podUID="72548522-bba4-4f29-92ea-4d41d72e88b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 18:51:45.759432 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:45.759393    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 18:51:45.759986 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:45.759461    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs" podUID="72548522-bba4-4f29-92ea-4d41d72e88b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 18:51:46.747388 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:46.747350    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 18:51:46.747740 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:46.747710    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs" podUID="72548522-bba4-4f29-92ea-4d41d72e88b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 18:51:47.749175 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:47.749141    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 18:51:47.749775 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:47.749743    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs" podUID="72548522-bba4-4f29-92ea-4d41d72e88b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 18:51:48.139162 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:48.139126    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:48.760386 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:48.760349    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 18:51:48.760914 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:48.760418    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs" podUID="72548522-bba4-4f29-92ea-4d41d72e88b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 18:51:49.744369 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:49.744338    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 18:51:49.744567 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:49.744392    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs" podUID="72548522-bba4-4f29-92ea-4d41d72e88b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 18:51:50.749056 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:50.749025    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:50.749532 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:50.749415    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:50.750060 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:50.750038    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 18:51:51.992522 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:51.992462    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:52.933560 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:52.933521    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:57.746020 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:51:57.745984    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:01.269025 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.268984    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"]
Oct 30 18:52:01.273032 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.273015    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:01.340874 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.340831    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-fd61e3\"/\"kube-root-ca.crt\""
Oct 30 18:52:01.376808 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.376775    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-fd61e3\"/\"distinct-2-w1-fd61e3-kserve-self-signed-certs\""
Oct 30 18:52:01.376987 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.376834    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-fd61e3\"/\"default-dockercfg-rgk5f\""
Oct 30 18:52:01.379171 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.379147    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"]
Oct 30 18:52:01.385845 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.385817    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-model-cache\") pod \"distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") " pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:01.385975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.385939    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80773b82-666e-428b-aeec-361a6c9cd6ac-tls-certs\") pod \"distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") " pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:01.386036 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.386022    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-home\") pod \"distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") " pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:01.386082 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.386053    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-tmp-dir\") pod \"distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") " pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:01.386211 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.386191    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5rrk\" (UniqueName: \"kubernetes.io/projected/80773b82-666e-428b-aeec-361a6c9cd6ac-kube-api-access-l5rrk\") pod \"distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") " pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:01.386249 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.386223    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-dshm\") pod \"distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") " pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:01.394999 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.394977    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:01.443762 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.443726    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-fd61e3\"/\"openshift-service-ca.crt\""
Oct 30 18:52:01.487011 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.486978    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80773b82-666e-428b-aeec-361a6c9cd6ac-tls-certs\") pod \"distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") " pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:01.487210 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.487067    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-home\") pod \"distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") " pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:01.487210 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.487104    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-tmp-dir\") pod \"distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") " pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:01.487337 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.487232    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5rrk\" (UniqueName: \"kubernetes.io/projected/80773b82-666e-428b-aeec-361a6c9cd6ac-kube-api-access-l5rrk\") pod \"distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") " pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:01.487337 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.487263    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-dshm\") pod \"distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") " pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:01.487337 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.487317    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-model-cache\") pod \"distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") " pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:01.487557 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.487532    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-home\") pod \"distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") " pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:01.487636 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.487565    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-tmp-dir\") pod \"distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") " pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:01.487728 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.487709    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-model-cache\") pod \"distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") " pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:01.489440 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.489420    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-dshm\") pod \"distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") " pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:01.489765 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.489748    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80773b82-666e-428b-aeec-361a6c9cd6ac-tls-certs\") pod \"distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") " pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:01.548379 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.548354    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5rrk\" (UniqueName: \"kubernetes.io/projected/80773b82-666e-428b-aeec-361a6c9cd6ac-kube-api-access-l5rrk\") pod \"distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") " pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:01.585116 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:01.585085    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:02.020513 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:02.020486    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"]
Oct 30 18:52:02.022117 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:52:02.022094    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80773b82_666e_428b_aeec_361a6c9cd6ac.slice/crio-9038a18ca663dba7b37f6c6055afa79657d36eba2df6ec2831f36bb0be4c0307 WatchSource:0}: Error finding container 9038a18ca663dba7b37f6c6055afa79657d36eba2df6ec2831f36bb0be4c0307: Status 404 returned error can't find the container with id 9038a18ca663dba7b37f6c6055afa79657d36eba2df6ec2831f36bb0be4c0307
Oct 30 18:52:02.656665 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:02.656536    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:02.698326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:02.698292    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n" event={"ID":"80773b82-666e-428b-aeec-361a6c9cd6ac","Type":"ContainerStarted","Data":"12f9d851304b567522173c6e48d033d723d953a0a977bea14c94466e80c93d2e"}
Oct 30 18:52:02.698466 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:02.698340    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n" event={"ID":"80773b82-666e-428b-aeec-361a6c9cd6ac","Type":"ContainerStarted","Data":"9038a18ca663dba7b37f6c6055afa79657d36eba2df6ec2831f36bb0be4c0307"}
Oct 30 18:52:02.818182 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:02.818130    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n" podStartSLOduration=1.8181128420000001 podStartE2EDuration="1.818112842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:02.81727316 +0000 UTC m=+1043.529021858" watchObservedRunningTime="2026-10-30 18:52:02.818112842 +0000 UTC m=+1043.529861539"
Oct 30 18:52:03.585750 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:03.585709    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:07.682503 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:07.682470    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:08.204091 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:52:08.204052    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:52:11.586015 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:11.585981    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:11.598387 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:11.598364    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:11.763573 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:11.763543    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:52:12.592804 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:12.592765    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:13.634180 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:13.634145    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:13.821974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:13.821898    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:15.008590 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:15.008553    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:17.402426 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:17.402388    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:21.053179 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:21.053148    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:22.243912 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:22.243867    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:27.032445 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:27.032406    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:27.757252 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:27.757218    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-896496998-wbhwr"]
Oct 30 18:52:27.811976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:27.811892    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"]
Oct 30 18:52:27.833807 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:27.833776    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:27.913975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:27.907373    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:28.161976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.094799    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/142d862b-ea80-4125-b065-7c3d471fe400-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.188126 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.188080    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.201090 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.201054    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.216057 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.216023    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9lrq\" (UniqueName: \"kubernetes.io/projected/142d862b-ea80-4125-b065-7c3d471fe400-kube-api-access-r9lrq\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.216292 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.216273    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/142d862b-ea80-4125-b065-7c3d471fe400-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.216573 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.216549    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.216745 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.216724    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.216902 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.216884    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/142d862b-ea80-4125-b065-7c3d471fe400-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.217071 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.217053    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.337973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.328533    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"]
Oct 30 18:52:28.337973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.329090    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:28.337973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.328694    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.337973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.329722    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.337973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.329995    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/142d862b-ea80-4125-b065-7c3d471fe400-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.337973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.330067    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.337973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.330116    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.337973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.330145    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9lrq\" (UniqueName: \"kubernetes.io/projected/142d862b-ea80-4125-b065-7c3d471fe400-kube-api-access-r9lrq\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.337973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.330204    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/142d862b-ea80-4125-b065-7c3d471fe400-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.337973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.330301    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.337973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.330349    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.337973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.330381    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/142d862b-ea80-4125-b065-7c3d471fe400-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.337973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.331244    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/142d862b-ea80-4125-b065-7c3d471fe400-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.337973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.331869    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.337973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.332184    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.337973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.333133    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.356227 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.356131    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.376121 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.376086    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/142d862b-ea80-4125-b065-7c3d471fe400-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.397692 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.397613    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9lrq\" (UniqueName: \"kubernetes.io/projected/142d862b-ea80-4125-b065-7c3d471fe400-kube-api-access-r9lrq\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.422536 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.422452    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/142d862b-ea80-4125-b065-7c3d471fe400-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.662286 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.662252    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:28.806107 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:28.806072    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:29.684765 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:29.683641    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw" podUID="1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:29.997027 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:29.993344    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"]
Oct 30 18:52:30.079975 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:52:30.079521    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod142d862b_ea80_4125_b065_7c3d471fe400.slice/crio-7b6f3a67d4633fe0501a9b15f222a69c0f64747aa50d3a94fcd4c3c378a59063 WatchSource:0}: Error finding container 7b6f3a67d4633fe0501a9b15f222a69c0f64747aa50d3a94fcd4c3c378a59063: Status 404 returned error can't find the container with id 7b6f3a67d4633fe0501a9b15f222a69c0f64747aa50d3a94fcd4c3c378a59063
Oct 30 18:52:30.106221 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:30.100761    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:52:30.106221 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:30.100872    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:52:30.106221 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:30.100921    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:52:30.303836 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:30.303804    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8" event={"ID":"142d862b-ea80-4125-b065-7c3d471fe400","Type":"ContainerStarted","Data":"7b6f3a67d4633fe0501a9b15f222a69c0f64747aa50d3a94fcd4c3c378a59063"}
Oct 30 18:52:31.310467 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:31.310430    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8" event={"ID":"142d862b-ea80-4125-b065-7c3d471fe400","Type":"ContainerStarted","Data":"492e2c395aae3fa51c232eaa9bc21adb8734e3a721846da663dcb9cf8976f621"}
Oct 30 18:52:31.531854 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:31.531797    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8" podStartSLOduration=4.531779369 podStartE2EDuration="4.531779369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:31.480222095 +0000 UTC m=+1072.191970793" watchObservedRunningTime="2026-10-30 18:52:31.531779369 +0000 UTC m=+1072.243528067"
Oct 30 18:52:31.663858 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:31.663767    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:31.665723 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:31.665697    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 18:52:31.665869 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:31.665747    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8" podUID="142d862b-ea80-4125-b065-7c3d471fe400" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 18:52:32.157987 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:32.157932    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:32.663759 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:32.663717    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 18:52:32.664204 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:32.663785    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8" podUID="142d862b-ea80-4125-b065-7c3d471fe400" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 18:52:32.759783 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:32.759075    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr" podUID="87a24e2c-6c3e-4203-9030-e009165d27a5" containerName="payload-processing" containerID="cri-o://c97d335ce9163374c5cdc6e32ca01804f763fb01656f0fa381205ffa1d11dbbc" gracePeriod=25
Oct 30 18:52:33.059347 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.059307    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr"
Oct 30 18:52:33.105428 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.105397    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87a24e2c-6c3e-4203-9030-e009165d27a5-kube-api-access-2zmnl\" (UniqueName: \"kubernetes.io/projected/87a24e2c-6c3e-4203-9030-e009165d27a5-kube-api-access-2zmnl\") pod \"87a24e2c-6c3e-4203-9030-e009165d27a5\" (UID: \"87a24e2c-6c3e-4203-9030-e009165d27a5\") "
Oct 30 18:52:33.105606 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.105464    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/87a24e2c-6c3e-4203-9030-e009165d27a5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87a24e2c-6c3e-4203-9030-e009165d27a5-plugins-config-volume\") pod \"87a24e2c-6c3e-4203-9030-e009165d27a5\" (UID: \"87a24e2c-6c3e-4203-9030-e009165d27a5\") "
Oct 30 18:52:33.105606 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.105557    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87a24e2c-6c3e-4203-9030-e009165d27a5-provider-credentials\" (UniqueName: \"kubernetes.io/projected/87a24e2c-6c3e-4203-9030-e009165d27a5-provider-credentials\") pod \"87a24e2c-6c3e-4203-9030-e009165d27a5\" (UID: \"87a24e2c-6c3e-4203-9030-e009165d27a5\") "
Oct 30 18:52:33.105720 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.105665    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/87a24e2c-6c3e-4203-9030-e009165d27a5-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/87a24e2c-6c3e-4203-9030-e009165d27a5-routing-overlay-volume\") pod \"87a24e2c-6c3e-4203-9030-e009165d27a5\" (UID: \"87a24e2c-6c3e-4203-9030-e009165d27a5\") "
Oct 30 18:52:33.105874 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.105842    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87a24e2c-6c3e-4203-9030-e009165d27a5-plugins-config-volume" pod "87a24e2c-6c3e-4203-9030-e009165d27a5" (UID: "87a24e2c-6c3e-4203-9030-e009165d27a5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:33.106064 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.106043    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87a24e2c-6c3e-4203-9030-e009165d27a5-routing-overlay-volume" pod "87a24e2c-6c3e-4203-9030-e009165d27a5" (UID: "87a24e2c-6c3e-4203-9030-e009165d27a5"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:33.106217 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.106200    5820 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/87a24e2c-6c3e-4203-9030-e009165d27a5-routing-overlay-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.106453 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.106218    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87a24e2c-6c3e-4203-9030-e009165d27a5-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.107738 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.107707    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87a24e2c-6c3e-4203-9030-e009165d27a5-kube-api-access-2zmnl" pod "87a24e2c-6c3e-4203-9030-e009165d27a5" (UID: "87a24e2c-6c3e-4203-9030-e009165d27a5"). InnerVolumeSpecName "kube-api-access-2zmnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:33.107835 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.107797    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87a24e2c-6c3e-4203-9030-e009165d27a5-provider-credentials" pod "87a24e2c-6c3e-4203-9030-e009165d27a5" (UID: "87a24e2c-6c3e-4203-9030-e009165d27a5"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:33.208145 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.208064    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zmnl\" (UniqueName: \"kubernetes.io/projected/87a24e2c-6c3e-4203-9030-e009165d27a5-kube-api-access-2zmnl\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.208145 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.208102    5820 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/87a24e2c-6c3e-4203-9030-e009165d27a5-provider-credentials\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.332638 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.332602    5820 generic.go:356] "Generic (PLEG): container finished" podID="87a24e2c-6c3e-4203-9030-e009165d27a5" containerID="c97d335ce9163374c5cdc6e32ca01804f763fb01656f0fa381205ffa1d11dbbc" exitCode=0
Oct 30 18:52:33.332799 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.332704    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr"
Oct 30 18:52:33.332799 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.332717    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr" event={"ID":"87a24e2c-6c3e-4203-9030-e009165d27a5","Type":"ContainerDied","Data":"c97d335ce9163374c5cdc6e32ca01804f763fb01656f0fa381205ffa1d11dbbc"}
Oct 30 18:52:33.332799 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.332783    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-896496998-wbhwr" event={"ID":"87a24e2c-6c3e-4203-9030-e009165d27a5","Type":"ContainerDied","Data":"893fdca585c6180ed5044d0cfecf98813cefa44245d5aed472761d7e3e53b1ec"}
Oct 30 18:52:33.332923 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.332805    5820 scope.go:122] "RemoveContainer" containerID="c97d335ce9163374c5cdc6e32ca01804f763fb01656f0fa381205ffa1d11dbbc"
Oct 30 18:52:33.371819 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.371794    5820 scope.go:122] "RemoveContainer" containerID="c97d335ce9163374c5cdc6e32ca01804f763fb01656f0fa381205ffa1d11dbbc"
Oct 30 18:52:33.372885 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:52:33.372850    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c97d335ce9163374c5cdc6e32ca01804f763fb01656f0fa381205ffa1d11dbbc\": container with ID starting with c97d335ce9163374c5cdc6e32ca01804f763fb01656f0fa381205ffa1d11dbbc not found: ID does not exist" containerID="c97d335ce9163374c5cdc6e32ca01804f763fb01656f0fa381205ffa1d11dbbc"
Oct 30 18:52:33.373110 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.373079    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c97d335ce9163374c5cdc6e32ca01804f763fb01656f0fa381205ffa1d11dbbc"} err="failed to get container status \"c97d335ce9163374c5cdc6e32ca01804f763fb01656f0fa381205ffa1d11dbbc\": rpc error: code = NotFound desc = could not find container \"c97d335ce9163374c5cdc6e32ca01804f763fb01656f0fa381205ffa1d11dbbc\": container with ID starting with c97d335ce9163374c5cdc6e32ca01804f763fb01656f0fa381205ffa1d11dbbc not found: ID does not exist"
Oct 30 18:52:33.416595 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:52:33.416564    5820 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87a24e2c_6c3e_4203_9030_e009165d27a5.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 18:52:33.515281 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.515210    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-896496998-wbhwr"]
Oct 30 18:52:33.519128 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.519101    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-896496998-wbhwr"]
Oct 30 18:52:33.663689 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.663652    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 18:52:33.663874 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:33.663719    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8" podUID="142d862b-ea80-4125-b065-7c3d471fe400" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 18:52:34.663470 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:34.663432    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 18:52:34.663664 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:34.663493    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8" podUID="142d862b-ea80-4125-b065-7c3d471fe400" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 18:52:35.457976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:35.457747    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="87a24e2c-6c3e-4203-9030-e009165d27a5" path="/var/lib/kubelet/pods/87a24e2c-6c3e-4203-9030-e009165d27a5/volumes"
Oct 30 18:52:35.663480 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:35.663451    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 18:52:35.663652 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:35.663504    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8" podUID="142d862b-ea80-4125-b065-7c3d471fe400" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 18:52:36.663838 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:36.663803    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 18:52:36.664307 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:36.663863    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8" podUID="142d862b-ea80-4125-b065-7c3d471fe400" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 18:52:36.964680 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:36.964587    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:37.667156 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:37.667124    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:37.667620 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:37.667402    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:37.668048 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:37.668028    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 18:52:38.203237 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:52:38.203205    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:52:38.518039 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:38.517939    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:42.372147 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:42.372099    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:45.390785 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:45.390745    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:47.238974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:47.235766    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:52.042167 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:52.042128    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:52.346406 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:52.346372    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:56.963924 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:56.963885    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:57.198194 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:52:57.198159    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:02.131582 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:02.130609    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:06.967551 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:06.967514    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:07.571838 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.571806    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"]
Oct 30 18:53:07.572463 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.572443    5820 cpu_manager.go:414] "containerMap: removing container" podUID="87a24e2c-6c3e-4203-9030-e009165d27a5" containerName="payload-processing"
Oct 30 18:53:07.572463 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.572462    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="87a24e2c-6c3e-4203-9030-e009165d27a5" containerName="payload-processing"
Oct 30 18:53:07.572613 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.572553    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="87a24e2c-6c3e-4203-9030-e009165d27a5" containerName="payload-processing"
Oct 30 18:53:07.576091 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.576077    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.623222 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.623185    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"]
Oct 30 18:53:07.690085 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.690050    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-credential-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.690249 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.690126    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-workload-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.690249 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.690152    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-envoy\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.690330 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.690284    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-podinfo\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.690330 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.690312    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-token\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.690417 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.690339    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-workload-certs\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.690470 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.690432    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istiod-ca-cert\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.690470 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.690450    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgtw8\" (UniqueName: \"kubernetes.io/projected/180fc1d4-d421-446a-afd7-f1f59bfafe0f-kube-api-access-zgtw8\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.690470 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.690468    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-data\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.801971 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.801760    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-workload-certs\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.801971 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.801890    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istiod-ca-cert\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.801971 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.801917    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgtw8\" (UniqueName: \"kubernetes.io/projected/180fc1d4-d421-446a-afd7-f1f59bfafe0f-kube-api-access-zgtw8\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.801971 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.801964    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-data\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.802345 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.802055    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-credential-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.802345 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.802087    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-workload-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.802345 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.802113    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-envoy\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.802345 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.802243    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-podinfo\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.802345 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.802277    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-token\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.805970 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.803016    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-workload-certs\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.805970 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.803639    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istiod-ca-cert\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.805970 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.804119    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-data\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.805970 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.804365    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-credential-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.805970 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.804600    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-workload-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.832493 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.832412    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-podinfo\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.840048 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.833699    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-envoy\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.863447 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.863402    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-token\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.864128 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.864109    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgtw8\" (UniqueName: \"kubernetes.io/projected/180fc1d4-d421-446a-afd7-f1f59bfafe0f-kube-api-access-zgtw8\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:07.889978 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:07.889925    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:08.288209 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:53:08.287476    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:53:08.394019 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:08.393985    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"]
Oct 30 18:53:08.397836 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:53:08.396816    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod180fc1d4_d421_446a_afd7_f1f59bfafe0f.slice/crio-d9f76d8de9b54af4f988d82336a46b4878e8badf5568f01505af256436760d72 WatchSource:0}: Error finding container d9f76d8de9b54af4f988d82336a46b4878e8badf5568f01505af256436760d72: Status 404 returned error can't find the container with id d9f76d8de9b54af4f988d82336a46b4878e8badf5568f01505af256436760d72
Oct 30 18:53:08.413265 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:08.413183    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:53:08.413458 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:08.413391    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:53:08.413533 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:08.413490    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:53:08.607026 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:08.604070    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48" event={"ID":"180fc1d4-d421-446a-afd7-f1f59bfafe0f","Type":"ContainerStarted","Data":"9ee0872e94052e046652ebecab0c991b075fc7ab7a2e18814f9eea958d629189"}
Oct 30 18:53:08.607026 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:08.604180    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48" event={"ID":"180fc1d4-d421-446a-afd7-f1f59bfafe0f","Type":"ContainerStarted","Data":"d9f76d8de9b54af4f988d82336a46b4878e8badf5568f01505af256436760d72"}
Oct 30 18:53:08.891029 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:08.890795    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:09.891976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:09.891925    5820 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 18:53:09.892494 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:09.891997    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48" podUID="180fc1d4-d421-446a-afd7-f1f59bfafe0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 18:53:10.891232 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:10.891202    5820 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 18:53:10.891384 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:10.891256    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48" podUID="180fc1d4-d421-446a-afd7-f1f59bfafe0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 18:53:11.890372 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:11.890333    5820 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 18:53:11.890783 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:11.890393    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48" podUID="180fc1d4-d421-446a-afd7-f1f59bfafe0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 18:53:11.947777 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:11.946683    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:12.891249 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:12.891208    5820 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 18:53:12.891667 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:12.891271    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48" podUID="180fc1d4-d421-446a-afd7-f1f59bfafe0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 18:53:13.853069 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:13.853002    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48" podStartSLOduration=6.852981665 podStartE2EDuration="6.852981665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:08.730145849 +0000 UTC m=+1109.441894548" watchObservedRunningTime="2026-10-30 18:53:13.852981665 +0000 UTC m=+1114.564730368"
Oct 30 18:53:13.857986 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:13.855146    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"]
Oct 30 18:53:13.867200 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:13.867174    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:13.891080 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:13.891048    5820 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 18:53:13.891246 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:13.891103    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48" podUID="180fc1d4-d421-446a-afd7-f1f59bfafe0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 18:53:13.918136 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:13.918107    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"]
Oct 30 18:53:13.965012 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:13.964978    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-4a7a52\"/\"distinct-w2-4a7a52-kserve-self-signed-certs\""
Oct 30 18:53:13.965012 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:13.965000    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-4a7a52\"/\"default-dockercfg-qzpjw\""
Oct 30 18:53:13.978241 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:13.978212    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:14.013054 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.013016    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-4a7a52\"/\"openshift-service-ca.crt\""
Oct 30 18:53:14.013255 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.013178    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-4a7a52\"/\"kube-root-ca.crt\""
Oct 30 18:53:14.020771 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.020720    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmtd7\" (UniqueName: \"kubernetes.io/projected/7300a782-cf82-48ef-a0da-0429b80a3480-kube-api-access-mmtd7\") pod \"distinct-w2-4a7a52-kserve-867d67787-qdjqk\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:14.020771 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.020765    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-tmp-dir\") pod \"distinct-w2-4a7a52-kserve-867d67787-qdjqk\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:14.021051 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.020801    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7300a782-cf82-48ef-a0da-0429b80a3480-tls-certs\") pod \"distinct-w2-4a7a52-kserve-867d67787-qdjqk\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:14.021051 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.020860    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-home\") pod \"distinct-w2-4a7a52-kserve-867d67787-qdjqk\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:14.021051 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.020914    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-model-cache\") pod \"distinct-w2-4a7a52-kserve-867d67787-qdjqk\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:14.021051 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.021015    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-dshm\") pod \"distinct-w2-4a7a52-kserve-867d67787-qdjqk\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:14.122716 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.122639    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7300a782-cf82-48ef-a0da-0429b80a3480-tls-certs\") pod \"distinct-w2-4a7a52-kserve-867d67787-qdjqk\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:14.122716 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.122687    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-home\") pod \"distinct-w2-4a7a52-kserve-867d67787-qdjqk\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:14.122942 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.122720    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-model-cache\") pod \"distinct-w2-4a7a52-kserve-867d67787-qdjqk\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:14.122942 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.122768    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-dshm\") pod \"distinct-w2-4a7a52-kserve-867d67787-qdjqk\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:14.123224 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.123202    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-model-cache\") pod \"distinct-w2-4a7a52-kserve-867d67787-qdjqk\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:14.123331 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.123307    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-home\") pod \"distinct-w2-4a7a52-kserve-867d67787-qdjqk\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:14.123331 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.123320    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmtd7\" (UniqueName: \"kubernetes.io/projected/7300a782-cf82-48ef-a0da-0429b80a3480-kube-api-access-mmtd7\") pod \"distinct-w2-4a7a52-kserve-867d67787-qdjqk\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:14.123442 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.123348    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-tmp-dir\") pod \"distinct-w2-4a7a52-kserve-867d67787-qdjqk\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:14.123650 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.123631    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-tmp-dir\") pod \"distinct-w2-4a7a52-kserve-867d67787-qdjqk\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:14.125330 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.125301    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-dshm\") pod \"distinct-w2-4a7a52-kserve-867d67787-qdjqk\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:14.125870 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.125848    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7300a782-cf82-48ef-a0da-0429b80a3480-tls-certs\") pod \"distinct-w2-4a7a52-kserve-867d67787-qdjqk\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:14.184545 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.184514    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmtd7\" (UniqueName: \"kubernetes.io/projected/7300a782-cf82-48ef-a0da-0429b80a3480-kube-api-access-mmtd7\") pod \"distinct-w2-4a7a52-kserve-867d67787-qdjqk\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:14.484196 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.484112    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:14.890054 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.890025    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"]
Oct 30 18:53:14.890373 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.890345    5820 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 18:53:14.890489 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:14.890407    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48" podUID="180fc1d4-d421-446a-afd7-f1f59bfafe0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 18:53:14.892401 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:53:14.892373    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7300a782_cf82_48ef_a0da_0429b80a3480.slice/crio-2e7a4d85199813fc304bae01aa79656c096989d709866c46642ca69f2bd7961d WatchSource:0}: Error finding container 2e7a4d85199813fc304bae01aa79656c096989d709866c46642ca69f2bd7961d: Status 404 returned error can't find the container with id 2e7a4d85199813fc304bae01aa79656c096989d709866c46642ca69f2bd7961d
Oct 30 18:53:15.681095 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:15.681060    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk" event={"ID":"7300a782-cf82-48ef-a0da-0429b80a3480","Type":"ContainerStarted","Data":"34318dabe902981b54edd50a43d1255c77e9bcad416a5d6ea9c2b9a165538923"}
Oct 30 18:53:15.681562 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:15.681103    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk" event={"ID":"7300a782-cf82-48ef-a0da-0429b80a3480","Type":"ContainerStarted","Data":"2e7a4d85199813fc304bae01aa79656c096989d709866c46642ca69f2bd7961d"}
Oct 30 18:53:15.802110 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:15.802056    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk" podStartSLOduration=2.802041688 podStartE2EDuration="2.802041688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:15.799686726 +0000 UTC m=+1116.511435423" watchObservedRunningTime="2026-10-30 18:53:15.802041688 +0000 UTC m=+1116.513790393"
Oct 30 18:53:15.894185 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:15.894159    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:15.894469 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:15.894449    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:15.895191 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:15.895171    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 18:53:16.064814 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:16.064786    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:16.257501 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:16.257460    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:16.484850 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:16.484775    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:16.735838 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:16.732813    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:21.516605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:21.516566    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:23.949974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:23.948204    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:24.484749 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:24.484715    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:24.497799 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:24.497775    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:24.745774 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:24.745692    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:53:26.389235 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:26.389117    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:28.284936 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.284903    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"]
Oct 30 18:53:28.288981 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.288965    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.338284 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.338250    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.338284 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.338283    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.338529 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.338302    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjgwk\" (UniqueName: \"kubernetes.io/projected/29ce830e-8096-4239-8ad4-03163e8fc5dc-kube-api-access-hjgwk\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.338529 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.338469    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.338615 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.338566    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.338615 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.338591    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.338615 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.338608    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29ce830e-8096-4239-8ad4-03163e8fc5dc-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.338747 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.338673    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.338747 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.338687    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.439821 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.439785    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.439821 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.439822    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.440055 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.439843    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjgwk\" (UniqueName: \"kubernetes.io/projected/29ce830e-8096-4239-8ad4-03163e8fc5dc-kube-api-access-hjgwk\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.440055 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.439936    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.440055 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.440016    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.440055 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.440038    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.440259 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.440059    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29ce830e-8096-4239-8ad4-03163e8fc5dc-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.440259 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.440130    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.440259 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.440157    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.440259 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.440251    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.440501 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.440479    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.440556 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.440538    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.440593 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.440570    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.440787 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.440767    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29ce830e-8096-4239-8ad4-03163e8fc5dc-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.442565 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.442537    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.442937 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.442919    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.501091 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.501055    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjgwk\" (UniqueName: \"kubernetes.io/projected/29ce830e-8096-4239-8ad4-03163e8fc5dc-kube-api-access-hjgwk\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.501238 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.501104    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:28.514195 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.514163    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"]
Oct 30 18:53:28.602991 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:28.602937    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:29.038155 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.038133    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"]
Oct 30 18:53:29.040035 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:53:29.040007    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29ce830e_8096_4239_8ad4_03163e8fc5dc.slice/crio-5157de500baa93d526308e8dc72e047d7091f23bb230a8c0e54517828452446a WatchSource:0}: Error finding container 5157de500baa93d526308e8dc72e047d7091f23bb230a8c0e54517828452446a: Status 404 returned error can't find the container with id 5157de500baa93d526308e8dc72e047d7091f23bb230a8c0e54517828452446a
Oct 30 18:53:29.042717 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.042685    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:53:29.042802 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.042749    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:53:29.042802 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.042778    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:53:29.720989 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.720937    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"]
Oct 30 18:53:29.725077 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.725056    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.770326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.770285    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796" event={"ID":"29ce830e-8096-4239-8ad4-03163e8fc5dc","Type":"ContainerStarted","Data":"def293b92810a87ddde97b30ad90ddcff6ebda969bf0871b9dda302d9c74cf69"}
Oct 30 18:53:29.772016 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.770351    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796" event={"ID":"29ce830e-8096-4239-8ad4-03163e8fc5dc","Type":"ContainerStarted","Data":"5157de500baa93d526308e8dc72e047d7091f23bb230a8c0e54517828452446a"}
Oct 30 18:53:29.775294 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.775265    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-credential-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.775416 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.775319    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn7vn\" (UniqueName: \"kubernetes.io/projected/0378c741-36a6-4c68-97bb-f3c6f4ec3470-kube-api-access-kn7vn\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.775486 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.775461    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-workload-certs\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.775540 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.775524    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-envoy\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.775687 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.775671    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istiod-ca-cert\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.775766 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.775750    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-podinfo\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.775820 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.775781    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-data\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.775910 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.775888    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-workload-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.776014 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.775924    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-token\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.786240 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.786217    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"]
Oct 30 18:53:29.876682 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.876648    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-podinfo\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.876682 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.876686    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-data\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.876968 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.876730    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-workload-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.876968 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.876751    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-token\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.876968 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.876789    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-credential-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.876968 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.876831    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn7vn\" (UniqueName: \"kubernetes.io/projected/0378c741-36a6-4c68-97bb-f3c6f4ec3470-kube-api-access-kn7vn\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.876968 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.876863    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-workload-certs\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.876968 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.876899    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-envoy\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.877358 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.877017    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istiod-ca-cert\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.878101 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.878071    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-data\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.878223 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.878154    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-credential-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.878286 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.878227    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-workload-certs\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.878286 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.878281    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-workload-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.878476 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.878455    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istiod-ca-cert\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.880279 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.880258    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-envoy\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.880797 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.880772    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-podinfo\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.960609 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.960572    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-token\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.961341 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.961319    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn7vn\" (UniqueName: \"kubernetes.io/projected/0378c741-36a6-4c68-97bb-f3c6f4ec3470-kube-api-access-kn7vn\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:29.981606 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:29.981522    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796" podStartSLOduration=1.98150787 podStartE2EDuration="1.98150787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:29.979002605 +0000 UTC m=+1130.690751302" watchObservedRunningTime="2026-10-30 18:53:29.98150787 +0000 UTC m=+1130.693256566"
Oct 30 18:53:30.063050 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:30.063014    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:30.501437 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:30.501409    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"]
Oct 30 18:53:30.502899 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:53:30.502874    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0378c741_36a6_4c68_97bb_f3c6f4ec3470.slice/crio-48e8e9968c3c8dc0dfda3c214f3988067e36bed1f66ddb1c1ad1279fdae5987b WatchSource:0}: Error finding container 48e8e9968c3c8dc0dfda3c214f3988067e36bed1f66ddb1c1ad1279fdae5987b: Status 404 returned error can't find the container with id 48e8e9968c3c8dc0dfda3c214f3988067e36bed1f66ddb1c1ad1279fdae5987b
Oct 30 18:53:30.505396 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:30.505359    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:53:30.505472 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:30.505429    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:53:30.505472 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:30.505464    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:53:30.603832 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:30.603809    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:30.605255 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:30.605235    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 18:53:30.605345 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:30.605275    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796" podUID="29ce830e-8096-4239-8ad4-03163e8fc5dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 18:53:30.778799 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:30.778699    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc" event={"ID":"0378c741-36a6-4c68-97bb-f3c6f4ec3470","Type":"ContainerStarted","Data":"4e0cd1fe656371a287b55fd9340a0b73a1b8faa9570caf654416b39c35bac92c"}
Oct 30 18:53:30.778799 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:30.778765    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc" event={"ID":"0378c741-36a6-4c68-97bb-f3c6f4ec3470","Type":"ContainerStarted","Data":"48e8e9968c3c8dc0dfda3c214f3988067e36bed1f66ddb1c1ad1279fdae5987b"}
Oct 30 18:53:30.900598 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:30.900540    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc" podStartSLOduration=1.9005217989999998 podStartE2EDuration="1.900521799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:30.89831826 +0000 UTC m=+1131.610066958" watchObservedRunningTime="2026-10-30 18:53:30.900521799 +0000 UTC m=+1131.612270497"
Oct 30 18:53:31.066040 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:31.065321    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:31.068067 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:31.068031    5820 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 18:53:31.068214 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:31.068103    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc" podUID="0378c741-36a6-4c68-97bb-f3c6f4ec3470" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 18:53:31.507305 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:31.507194    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:31.609989 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:31.608220    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 18:53:31.609989 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:31.608280    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796" podUID="29ce830e-8096-4239-8ad4-03163e8fc5dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 18:53:32.083978 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:32.078099    5820 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 18:53:32.083978 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:32.078179    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc" podUID="0378c741-36a6-4c68-97bb-f3c6f4ec3470" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 18:53:32.604014 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:32.603978    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 18:53:32.604221 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:32.604038    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796" podUID="29ce830e-8096-4239-8ad4-03163e8fc5dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 18:53:33.065744 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:33.065312    5820 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 18:53:33.065744 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:33.065370    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc" podUID="0378c741-36a6-4c68-97bb-f3c6f4ec3470" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 18:53:33.604294 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:33.604259    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 18:53:33.604721 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:33.604309    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796" podUID="29ce830e-8096-4239-8ad4-03163e8fc5dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 18:53:34.087023 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:34.064523    5820 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 18:53:34.087023 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:34.064614    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc" podUID="0378c741-36a6-4c68-97bb-f3c6f4ec3470" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 18:53:34.603363 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:34.603328    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 18:53:34.603559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:34.603380    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796" podUID="29ce830e-8096-4239-8ad4-03163e8fc5dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 18:53:35.063861 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:35.063820    5820 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 18:53:35.064310 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:35.063890    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc" podUID="0378c741-36a6-4c68-97bb-f3c6f4ec3470" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 18:53:35.603661 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:35.603621    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 18:53:35.603846 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:35.603691    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796" podUID="29ce830e-8096-4239-8ad4-03163e8fc5dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 18:53:36.063505 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:36.063473    5820 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 18:53:36.063827 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:36.063782    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc" podUID="0378c741-36a6-4c68-97bb-f3c6f4ec3470" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 18:53:36.607818 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:36.607785    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:36.608237 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:36.608088    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:36.608798 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:36.608781    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 18:53:36.914713 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:36.914601    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:37.064297 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:37.064257    5820 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 18:53:37.064487 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:37.064340    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc" podUID="0378c741-36a6-4c68-97bb-f3c6f4ec3470" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 18:53:38.068107 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:38.068076    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:38.068582 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:38.068370    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:38.069198 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:38.069181    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 18:53:38.203531 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:53:38.203495    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:53:41.975993 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:41.973985    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:43.413748 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.413714    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"]
Oct 30 18:53:43.419002 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.418981    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.491920 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.491885    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.491920 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.491921    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/036d7b93-15ba-499b-9ed5-a4217c856742-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.492153 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.492073    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kc9t\" (UniqueName: \"kubernetes.io/projected/036d7b93-15ba-499b-9ed5-a4217c856742-kube-api-access-8kc9t\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.492153 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.492105    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.492153 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.492128    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.492276 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.492212    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.492276 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.492255    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/036d7b93-15ba-499b-9ed5-a4217c856742-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.492349 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.492277    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.492395 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.492375    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/036d7b93-15ba-499b-9ed5-a4217c856742-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.523703 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.523670    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"]
Oct 30 18:53:43.593554 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.593519    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.593554 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.593556    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.593763 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.593603    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.593763 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.593672    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/036d7b93-15ba-499b-9ed5-a4217c856742-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.593763 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.593701    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.593763 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.593732    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/036d7b93-15ba-499b-9ed5-a4217c856742-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.593982 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.593827    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.593982 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.593860    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/036d7b93-15ba-499b-9ed5-a4217c856742-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.594105 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.593981    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.594105 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.594007    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kc9t\" (UniqueName: \"kubernetes.io/projected/036d7b93-15ba-499b-9ed5-a4217c856742-kube-api-access-8kc9t\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.594105 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.594080    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.594250 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.594232    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.594515 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.594491    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.594607 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.594532    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/036d7b93-15ba-499b-9ed5-a4217c856742-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.596111 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.596090    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.596463 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.596443    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/036d7b93-15ba-499b-9ed5-a4217c856742-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.654574 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.654540    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/036d7b93-15ba-499b-9ed5-a4217c856742-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.654747 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.654591    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kc9t\" (UniqueName: \"kubernetes.io/projected/036d7b93-15ba-499b-9ed5-a4217c856742-kube-api-access-8kc9t\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:43.733063 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:43.732988    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:44.167747 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:44.167716    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"]
Oct 30 18:53:44.169242 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:53:44.169215    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod036d7b93_15ba_499b_9ed5_a4217c856742.slice/crio-3e8f3d666d992cb31cbdafc3bb32b873f8d64da7bb81f98cf8ed75d26fd01013 WatchSource:0}: Error finding container 3e8f3d666d992cb31cbdafc3bb32b873f8d64da7bb81f98cf8ed75d26fd01013: Status 404 returned error can't find the container with id 3e8f3d666d992cb31cbdafc3bb32b873f8d64da7bb81f98cf8ed75d26fd01013
Oct 30 18:53:44.171564 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:44.171531    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:53:44.171651 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:44.171596    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:53:44.171651 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:44.171624    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:53:44.889129 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:44.889086    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw" event={"ID":"036d7b93-15ba-499b-9ed5-a4217c856742","Type":"ContainerStarted","Data":"56de44c8af394d2e1b66452336dfa3f45613827ca0ff184be4f5a080bf487366"}
Oct 30 18:53:44.889638 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:44.889145    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw" event={"ID":"036d7b93-15ba-499b-9ed5-a4217c856742","Type":"ContainerStarted","Data":"3e8f3d666d992cb31cbdafc3bb32b873f8d64da7bb81f98cf8ed75d26fd01013"}
Oct 30 18:53:45.064147 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:45.064090    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw" podStartSLOduration=2.06407424 podStartE2EDuration="2.06407424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:45.063309485 +0000 UTC m=+1145.775058185" watchObservedRunningTime="2026-10-30 18:53:45.06407424 +0000 UTC m=+1145.775822939"
Oct 30 18:53:45.739725 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:45.738786    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:45.747685 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:45.747243    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 18:53:45.747685 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:45.747404    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw" podUID="036d7b93-15ba-499b-9ed5-a4217c856742" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 18:53:46.630712 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.630672    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"]
Oct 30 18:53:46.642239 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.642209    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:46.696004 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.695973    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-4a7a52-openshift-default-dockercfg-9p592\""
Oct 30 18:53:46.717668 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.711538    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"]
Oct 30 18:53:46.771393 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.765422    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 18:53:46.771393 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.765487    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw" podUID="036d7b93-15ba-499b-9ed5-a4217c856742" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 18:53:46.814845 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.814262    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istiod-ca-cert\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:46.814845 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.814316    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-podinfo\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:46.814845 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.814368    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-envoy\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:46.814845 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.814409    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-workload-certs\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:46.814845 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.814444    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-credential-socket\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:46.814845 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.814478    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-data\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:46.814845 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.814567    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grhq5\" (UniqueName: \"kubernetes.io/projected/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-kube-api-access-grhq5\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:46.814845 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.814607    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-token\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:46.814845 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.814648    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-workload-socket\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:46.927046 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.926912    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grhq5\" (UniqueName: \"kubernetes.io/projected/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-kube-api-access-grhq5\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:46.927046 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.927027    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-token\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:46.927296 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.927072    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-workload-socket\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:46.935973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.935095    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istiod-ca-cert\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:46.935973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.935171    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-podinfo\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:46.935973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.935256    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-envoy\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:46.935973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.935316    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-workload-certs\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:46.935973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.935374    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-credential-socket\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:46.935973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.935435    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-data\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:46.978978 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.965488    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-workload-socket\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:46.978978 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.966141    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-data\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:47.005974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.998424    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-workload-certs\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:47.005974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:46.998715    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-credential-socket\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:47.007122 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:47.006715    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istiod-ca-cert\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:47.122990 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:47.121297    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-envoy\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:47.123746 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:47.123721    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-token\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:47.134986 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:47.130622    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grhq5\" (UniqueName: \"kubernetes.io/projected/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-kube-api-access-grhq5\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:47.161016 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:47.152642    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-podinfo\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:47.248026 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:47.245860    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:47.305076 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:47.299055    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:47.740927 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:47.740891    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 18:53:47.742569 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:47.741676    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw" podUID="036d7b93-15ba-499b-9ed5-a4217c856742" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 18:53:47.778060 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:47.777471    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n" podUID="80773b82-666e-428b-aeec-361a6c9cd6ac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 18:53:48.621919 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:48.621808    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"]
Oct 30 18:53:48.667476 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:48.665790    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:53:48.667476 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:48.665875    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:53:48.667476 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:48.665917    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:53:48.761136 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:48.744534    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 18:53:48.761136 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:48.744591    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw" podUID="036d7b93-15ba-499b-9ed5-a4217c856742" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 18:53:49.302726 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:49.302688    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b" event={"ID":"cfb02c85-4568-4d67-9dfa-2c5252c3a67b","Type":"ContainerStarted","Data":"d84e87d80cf51fa0b067b2f4394d8eed3857a5ca1db7e334cf42b0103db7b955"}
Oct 30 18:53:49.302907 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:49.302753    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b" event={"ID":"cfb02c85-4568-4d67-9dfa-2c5252c3a67b","Type":"ContainerStarted","Data":"e2232e317dbfb69f2e6c6b2e6e6753fae4d1a792072b94ac8ae360b3f198dbcf"}
Oct 30 18:53:49.424921 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:49.424866    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b" podStartSLOduration=3.424848833 podStartE2EDuration="3.424848833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:49.422237219 +0000 UTC m=+1150.133985916" watchObservedRunningTime="2026-10-30 18:53:49.424848833 +0000 UTC m=+1150.136597529"
Oct 30 18:53:49.734028 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:49.733883    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 18:53:49.734028 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:49.733942    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw" podUID="036d7b93-15ba-499b-9ed5-a4217c856742" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 18:53:50.300698 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:50.300657    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:50.302064 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:50.302035    5820 patch_prober.go:29] interesting pod/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 18:53:50.302206 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:50.302116    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b" podUID="cfb02c85-4568-4d67-9dfa-2c5252c3a67b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 18:53:50.733927 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:50.733839    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 18:53:50.733927 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:50.733888    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw" podUID="036d7b93-15ba-499b-9ed5-a4217c856742" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 18:53:51.300580 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:51.300540    5820 patch_prober.go:29] interesting pod/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 18:53:51.300771 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:51.300616    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b" podUID="cfb02c85-4568-4d67-9dfa-2c5252c3a67b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 18:53:51.738146 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:51.738048    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 18:53:51.738420 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:51.738388    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw" podUID="036d7b93-15ba-499b-9ed5-a4217c856742" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 18:53:52.231285 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:52.231248    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:52.300576 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:52.300538    5820 patch_prober.go:29] interesting pod/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 18:53:52.300917 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:52.300886    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b" podUID="cfb02c85-4568-4d67-9dfa-2c5252c3a67b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 18:53:52.695464 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:52.695428    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:53.300302 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:53.300270    5820 patch_prober.go:29] interesting pod/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 18:53:53.300493 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:53.300328    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b" podUID="cfb02c85-4568-4d67-9dfa-2c5252c3a67b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 18:53:53.768022 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:53.763918    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:53.804118 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:53.804081    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:53:53.804418 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:53.804372    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw" podUID="036d7b93-15ba-499b-9ed5-a4217c856742" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:53:54.285308 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:54.285276    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:54.285735 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:54.285708    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:54.286748 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:54.286725    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 18:53:54.300827 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:54.300799    5820 patch_prober.go:29] interesting pod/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 18:53:54.300969 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:54.300844    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b" podUID="cfb02c85-4568-4d67-9dfa-2c5252c3a67b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 18:53:55.300750 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:55.300712    5820 patch_prober.go:29] interesting pod/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 18:53:55.301185 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:55.300766    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b" podUID="cfb02c85-4568-4d67-9dfa-2c5252c3a67b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 18:53:56.304491 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:56.304463    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:56.304929 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:56.304742    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:56.305420 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:56.305399    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 18:53:57.071888 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:53:57.071855    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:01.879017 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:01.878977    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:07.667081 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:07.667031    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw" podUID="1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:07.671358 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:07.671327    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:54:07.671614 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:07.671588    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796" podUID="29ce830e-8096-4239-8ad4-03163e8fc5dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:07.738096 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:07.738034    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk" podUID="7300a782-cf82-48ef-a0da-0429b80a3480" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:07.926738 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:07.926655    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:07.994150 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:07.994106    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b" podUID="1cae2237-0916-415a-9059-b94e0d950b0a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:08.412389 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:54:08.412325    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:54:12.833587 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:12.833552    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:15.053596 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.053562    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"]
Oct 30 18:54:15.058766 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.058743    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.121503 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.121471    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-data\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.121685 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.121514    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-credential-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.121685 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.121610    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-workload-certs\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.121797 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.121713    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm57m\" (UniqueName: \"kubernetes.io/projected/ce7148f2-3098-45df-bf88-4edfc8117df0-kube-api-access-mm57m\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.121797 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.121757    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-podinfo\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.121797 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.121773    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce7148f2-3098-45df-bf88-4edfc8117df0-istiod-ca-cert\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.121904 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.121890    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-envoy\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.121938 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.121915    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-token\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.122041 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.122027    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-workload-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.159696 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.159668    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"]
Oct 30 18:54:15.223402 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.223340    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-envoy\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.223402 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.223378    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-token\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.223688 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.223458    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-workload-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.223688 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.223532    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-data\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.223688 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.223566    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-credential-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.223688 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.223624    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-workload-certs\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.223919 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.223690    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm57m\" (UniqueName: \"kubernetes.io/projected/ce7148f2-3098-45df-bf88-4edfc8117df0-kube-api-access-mm57m\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.223919 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.223726    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-podinfo\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.223919 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.223750    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce7148f2-3098-45df-bf88-4edfc8117df0-istiod-ca-cert\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.224904 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.224171    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-data\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.224904 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.224536    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce7148f2-3098-45df-bf88-4edfc8117df0-istiod-ca-cert\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.224904 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.224698    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-workload-certs\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.224904 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.224770    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-credential-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.224904 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.224869    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-workload-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.226069 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.226042    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-envoy\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.226991 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.226939    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-podinfo\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.286007 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.285974    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-token\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.286172 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.286010    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm57m\" (UniqueName: \"kubernetes.io/projected/ce7148f2-3098-45df-bf88-4edfc8117df0-kube-api-access-mm57m\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.372477 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.372393    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:15.809976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.809916    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"]
Oct 30 18:54:15.814854 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:54:15.814823    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce7148f2_3098_45df_bf88_4edfc8117df0.slice/crio-7e69ed11914b4e9446412dcbdc3eec69fb5598334c37e4c4712540343d3eb476 WatchSource:0}: Error finding container 7e69ed11914b4e9446412dcbdc3eec69fb5598334c37e4c4712540343d3eb476: Status 404 returned error can't find the container with id 7e69ed11914b4e9446412dcbdc3eec69fb5598334c37e4c4712540343d3eb476
Oct 30 18:54:15.818211 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.818167    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:54:15.818341 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.818256    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:54:15.818341 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:15.818305    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:54:16.575748 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:16.575712    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz" event={"ID":"ce7148f2-3098-45df-bf88-4edfc8117df0","Type":"ContainerStarted","Data":"629aed487806c57d49941e388cfb49277090de230e2f220c37c31969e494d85d"}
Oct 30 18:54:16.576239 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:16.575765    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz" event={"ID":"ce7148f2-3098-45df-bf88-4edfc8117df0","Type":"ContainerStarted","Data":"7e69ed11914b4e9446412dcbdc3eec69fb5598334c37e4c4712540343d3eb476"}
Oct 30 18:54:16.761885 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:16.761826    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz" podStartSLOduration=1.761712811 podStartE2EDuration="1.761712811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:16.758271536 +0000 UTC m=+1177.470020234" watchObservedRunningTime="2026-10-30 18:54:16.761712811 +0000 UTC m=+1177.473461513"
Oct 30 18:54:17.373488 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:17.373396    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:17.375070 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:17.375046    5820 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 18:54:17.375188 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:17.375095    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz" podUID="ce7148f2-3098-45df-bf88-4edfc8117df0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 18:54:17.693024 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:17.688344    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:18.373637 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:18.373598    5820 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 18:54:18.373830 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:18.373659    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz" podUID="ce7148f2-3098-45df-bf88-4edfc8117df0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 18:54:19.373671 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:19.373636    5820 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 18:54:19.374099 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:19.373691    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz" podUID="ce7148f2-3098-45df-bf88-4edfc8117df0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 18:54:20.318135 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:20.318100    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:20.372714 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:20.372679    5820 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 18:54:20.372890 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:20.372742    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz" podUID="ce7148f2-3098-45df-bf88-4edfc8117df0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 18:54:21.373653 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:21.373616    5820 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 18:54:21.374099 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:21.373670    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz" podUID="ce7148f2-3098-45df-bf88-4edfc8117df0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 18:54:21.865974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:21.862904    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:22.377975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:22.375382    5820 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 18:54:22.377975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:22.375450    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz" podUID="ce7148f2-3098-45df-bf88-4edfc8117df0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 18:54:22.755570 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:22.755484    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:23.373850 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:23.373354    5820 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 18:54:23.373850 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:23.373431    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz" podUID="ce7148f2-3098-45df-bf88-4edfc8117df0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 18:54:23.734045 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:23.733389    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw" podUID="1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Oct 30 18:54:23.771977 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:23.761828    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n" podUID="80773b82-666e-428b-aeec-361a6c9cd6ac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:23.775747 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:23.775699    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:54:23.776042 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:23.776004    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8" podUID="142d862b-ea80-4125-b065-7c3d471fe400" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:24.051373 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:24.051332    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b" podUID="1cae2237-0916-415a-9059-b94e0d950b0a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:24.623255 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:24.623222    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:24.623814 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:24.623780    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:24.624686 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:24.624666    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 18:54:28.029714 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:28.029672    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:32.844971 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:32.844798    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:36.410467 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:36.410422    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:36.584362 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:36.584320    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:36.918088 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:36.918053    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:37.578749 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:37.578704    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:37.760575 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:37.760541    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:37.964079 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:37.962546    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:38.141273 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:38.141239    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:38.205706 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:54:38.205669    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:54:42.127307 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.127276    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"]
Oct 30 18:54:42.131536 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.131519    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:42.185301 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.185271    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-4a7a52\"/\"distinct-2-w2-4a7a52-kserve-self-signed-certs\""
Oct 30 18:54:42.210006 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.209975    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"]
Oct 30 18:54:42.253969 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.253926    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-tmp-dir\") pod \"distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:42.254130 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.254070    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnz87\" (UniqueName: \"kubernetes.io/projected/e31dcc89-d59d-4532-bda4-f28eb3b91095-kube-api-access-wnz87\") pod \"distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:42.254207 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.254168    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-home\") pod \"distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:42.254323 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.254234    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e31dcc89-d59d-4532-bda4-f28eb3b91095-tls-certs\") pod \"distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:42.254323 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.254272    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-dshm\") pod \"distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:42.254399 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.254346    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-model-cache\") pod \"distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:42.269160 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.269132    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:42.367219 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.357810    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-model-cache\") pod \"distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:42.367219 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.357984    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-tmp-dir\") pod \"distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:42.367219 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.358050    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnz87\" (UniqueName: \"kubernetes.io/projected/e31dcc89-d59d-4532-bda4-f28eb3b91095-kube-api-access-wnz87\") pod \"distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:42.367219 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.358155    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-home\") pod \"distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:42.367219 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.358231    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e31dcc89-d59d-4532-bda4-f28eb3b91095-tls-certs\") pod \"distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:42.367219 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.358276    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-dshm\") pod \"distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:42.367219 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.359388    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-model-cache\") pod \"distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:42.367219 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.359618    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-tmp-dir\") pod \"distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:42.367219 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.359833    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-home\") pod \"distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:42.368823 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.368791    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-dshm\") pod \"distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:42.370618 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.370590    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e31dcc89-d59d-4532-bda4-f28eb3b91095-tls-certs\") pod \"distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:42.420185 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.420120    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnz87\" (UniqueName: \"kubernetes.io/projected/e31dcc89-d59d-4532-bda4-f28eb3b91095-kube-api-access-wnz87\") pod \"distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") " pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:42.443842 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.443817    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:42.493476 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.493438    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:42.692059 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.682502    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:42.838902 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.838878    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"]
Oct 30 18:54:42.840167 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:54:42.840135    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode31dcc89_d59d_4532_bda4_f28eb3b91095.slice/crio-850744d2d248cbf69b5da9c4009286475bea900a37579b32888a38d8b9ab55a0 WatchSource:0}: Error finding container 850744d2d248cbf69b5da9c4009286475bea900a37579b32888a38d8b9ab55a0: Status 404 returned error can't find the container with id 850744d2d248cbf69b5da9c4009286475bea900a37579b32888a38d8b9ab55a0
Oct 30 18:54:42.943888 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.943789    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt" event={"ID":"e31dcc89-d59d-4532-bda4-f28eb3b91095","Type":"ContainerStarted","Data":"b18a0bf5f177ef37de8a1bbb51dcfffd0e40743263838f758e3327031c778d08"}
Oct 30 18:54:42.943888 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:42.943858    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt" event={"ID":"e31dcc89-d59d-4532-bda4-f28eb3b91095","Type":"ContainerStarted","Data":"850744d2d248cbf69b5da9c4009286475bea900a37579b32888a38d8b9ab55a0"}
Oct 30 18:54:43.061341 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:43.050567    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:43.079974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:43.073290    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt" podStartSLOduration=1.073271606 podStartE2EDuration="1.073271606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:43.063604039 +0000 UTC m=+1203.775352772" watchObservedRunningTime="2026-10-30 18:54:43.073271606 +0000 UTC m=+1203.785020303"
Oct 30 18:54:43.443976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:43.443918    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:45.670975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:45.670034    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw" podUID="1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:45.741976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:45.741042    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk" podUID="7300a782-cf82-48ef-a0da-0429b80a3480" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:45.761977 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:45.756035    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n" podUID="80773b82-666e-428b-aeec-361a6c9cd6ac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:46.006035 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:46.005045    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b" podUID="1cae2237-0916-415a-9059-b94e0d950b0a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:46.038978 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:46.038032    5820 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b" podUID="1cae2237-0916-415a-9059-b94e0d950b0a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:46.167976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:46.167045    5820 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw" podUID="1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:46.675974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:46.674076    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw" podUID="1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:46.700979 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:46.688760    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:54:46.754977 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:46.742691    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk" podUID="7300a782-cf82-48ef-a0da-0429b80a3480" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:46.754977 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:46.743198    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:54:46.758972 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:46.758027    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n" podUID="80773b82-666e-428b-aeec-361a6c9cd6ac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:46.802998 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:46.777373    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:54:47.026091 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:47.015995    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b" podUID="1cae2237-0916-415a-9059-b94e0d950b0a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:47.026091 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:47.018278    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:54:47.151405 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:47.151046    5820 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:54:47.151405 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:47.151128    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48" podUID="180fc1d4-d421-446a-afd7-f1f59bfafe0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:47.218655 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:47.218620    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:47.546977 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:47.526684    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:47.694001 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:47.675133    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw" podUID="1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Oct 30 18:54:47.718832 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:47.718796    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr"]
Oct 30 18:54:47.732673 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:47.732642    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr"
Oct 30 18:54:47.767980 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:47.745382    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk" podUID="7300a782-cf82-48ef-a0da-0429b80a3480" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": context deadline exceeded"
Oct 30 18:54:47.767980 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:47.761031    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n" podUID="80773b82-666e-428b-aeec-361a6c9cd6ac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 18:54:47.800044 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:47.796299    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-66baf45c\""
Oct 30 18:54:47.800044 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:47.797086    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-66baf45c-dockercfg-8c5j9\""
Oct 30 18:54:47.874984 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:47.873851    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrcpr\" (UniqueName: \"kubernetes.io/projected/68a37d9e-512a-4271-8916-6af9d7b69bdc-kube-api-access-hrcpr\") pod \"payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr\" (UID: \"68a37d9e-512a-4271-8916-6af9d7b69bdc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr"
Oct 30 18:54:47.874984 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:47.874038    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68a37d9e-512a-4271-8916-6af9d7b69bdc-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr\" (UID: \"68a37d9e-512a-4271-8916-6af9d7b69bdc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr"
Oct 30 18:54:48.004983 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:47.980108    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68a37d9e-512a-4271-8916-6af9d7b69bdc-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr\" (UID: \"68a37d9e-512a-4271-8916-6af9d7b69bdc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr"
Oct 30 18:54:48.004983 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:47.980296    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrcpr\" (UniqueName: \"kubernetes.io/projected/68a37d9e-512a-4271-8916-6af9d7b69bdc-kube-api-access-hrcpr\") pod \"payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr\" (UID: \"68a37d9e-512a-4271-8916-6af9d7b69bdc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr"
Oct 30 18:54:48.004983 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:47.981326    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68a37d9e-512a-4271-8916-6af9d7b69bdc-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr\" (UID: \"68a37d9e-512a-4271-8916-6af9d7b69bdc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr"
Oct 30 18:54:48.026797 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:48.019035    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b" podUID="1cae2237-0916-415a-9059-b94e0d950b0a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:48.117185 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:48.117099    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr"]
Oct 30 18:54:48.118232 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:48.118210    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrcpr\" (UniqueName: \"kubernetes.io/projected/68a37d9e-512a-4271-8916-6af9d7b69bdc-kube-api-access-hrcpr\") pod \"payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr\" (UID: \"68a37d9e-512a-4271-8916-6af9d7b69bdc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr"
Oct 30 18:54:48.175207 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:48.175173    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr"
Oct 30 18:54:48.645980 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:48.638816    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 18:54:49.061319 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:49.061282    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 18:54:49.154767 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:49.154718    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw" podUID="1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Oct 30 18:54:49.554164 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:49.554120    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b" podUID="1cae2237-0916-415a-9059-b94e0d950b0a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 18:54:49.931030 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:49.921742    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:50.061148 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:50.061059    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk" podUID="7300a782-cf82-48ef-a0da-0429b80a3480" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:50.430734 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:50.430706    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 18:54:50.480389 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:50.480092    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n" podUID="80773b82-666e-428b-aeec-361a6c9cd6ac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:50.522897 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:50.522851    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr"]
Oct 30 18:54:50.588975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:50.576060    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw" podUID="1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Oct 30 18:54:50.628015 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:50.619032    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-2r7zc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:54:50.628015 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:50.619093    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc" podUID="fb366d57-3daf-4ffc-a3e8-e771542ae33f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:50.680060 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:50.674024    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:51.109978 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:51.082419    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 18:54:51.141971 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:51.141304    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-xttc4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:54:51.141971 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:51.141373    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4" podUID="a373b999-a0cb-4016-9415-e977bd058b1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:51.416291 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:51.416205    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr" event={"ID":"68a37d9e-512a-4271-8916-6af9d7b69bdc","Type":"ContainerStarted","Data":"89d2b45f26fdc0bf2d0307d40d5a7df9dceac3865b2c04a12659714bb2c87956"}
Oct 30 18:54:52.423362 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:52.423327    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr" event={"ID":"68a37d9e-512a-4271-8916-6af9d7b69bdc","Type":"ContainerStarted","Data":"ad715e528bf2eb4c05fe3a092d7d05d271a2b6aff7c3b353b516401013c70702"}
Oct 30 18:54:52.423766 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:52.423465    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr"
Oct 30 18:54:52.444895 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:52.444866    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:52.458068 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:52.458040    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:52.544112 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:52.544060    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr" podStartSLOduration=5.544045747 podStartE2EDuration="5.544045747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:52.542288153 +0000 UTC m=+1213.254036850" watchObservedRunningTime="2026-10-30 18:54:52.544045747 +0000 UTC m=+1213.255794474"
Oct 30 18:54:53.092237 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:53.092200    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:53.441071 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:53.440998    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:54:54.674974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:54.674447    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:54.920250 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:54.920217    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"]
Oct 30 18:54:54.924413 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:54.924389    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:54.941871 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:54.941842    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:54.941998 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:54.941900    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:54.942056 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:54.942003    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:54.942056 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:54.942042    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:54.942148 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:54.942070    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:54.942148 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:54.942088    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffn62\" (UniqueName: \"kubernetes.io/projected/98a5a772-0770-40c9-8a01-5b991fee55d4-kube-api-access-ffn62\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:54.942229 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:54.942202    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:54.942281 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:54.942264    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98a5a772-0770-40c9-8a01-5b991fee55d4-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:54.942392 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:54.942352    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:55.022498 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.022455    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"]
Oct 30 18:54:55.043492 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.043454    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:55.043656 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.043526    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:55.043656 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.043583    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:55.043656 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.043632    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:55.043840 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.043717    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffn62\" (UniqueName: \"kubernetes.io/projected/98a5a772-0770-40c9-8a01-5b991fee55d4-kube-api-access-ffn62\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:55.043840 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.043764    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:55.043996 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.043840    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98a5a772-0770-40c9-8a01-5b991fee55d4-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:55.043996 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.043880    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:55.044126 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.044063    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:55.044186 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.044157    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:55.044310 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.044289    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:55.044460 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.044436    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:55.044804 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.044685    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:55.044895 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.044771    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98a5a772-0770-40c9-8a01-5b991fee55d4-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:55.046462 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.046443    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:55.046674 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.046651    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:55.104585 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.104555    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:55.105009 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.104993    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffn62\" (UniqueName: \"kubernetes.io/projected/98a5a772-0770-40c9-8a01-5b991fee55d4-kube-api-access-ffn62\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:55.238910 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.238819    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:55.669550 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.669525    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"]
Oct 30 18:54:55.671103 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:54:55.671078    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98a5a772_0770_40c9_8a01_5b991fee55d4.slice/crio-a79d0c8cc97964e3984efb8016a489eb40927000e5b68ea9b155e856dde52960 WatchSource:0}: Error finding container a79d0c8cc97964e3984efb8016a489eb40927000e5b68ea9b155e856dde52960: Status 404 returned error can't find the container with id a79d0c8cc97964e3984efb8016a489eb40927000e5b68ea9b155e856dde52960
Oct 30 18:54:55.673550 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.673513    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:54:55.673628 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.673581    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:54:55.673628 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:55.673609    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:54:56.466249 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:56.466212    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv" event={"ID":"98a5a772-0770-40c9-8a01-5b991fee55d4","Type":"ContainerStarted","Data":"8ed8e4f60fd86b64e0d314cb04a00444d1256f43c49341a643981caf79ccee67"}
Oct 30 18:54:56.466656 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:56.466268    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv" event={"ID":"98a5a772-0770-40c9-8a01-5b991fee55d4","Type":"ContainerStarted","Data":"a79d0c8cc97964e3984efb8016a489eb40927000e5b68ea9b155e856dde52960"}
Oct 30 18:54:56.640728 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:56.637329    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv" podStartSLOduration=2.637308042 podStartE2EDuration="2.637308042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:56.634905011 +0000 UTC m=+1217.346653722" watchObservedRunningTime="2026-10-30 18:54:56.637308042 +0000 UTC m=+1217.349056740"
Oct 30 18:54:57.059754 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:57.059715    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:57.253296 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:57.253265    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:54:57.260056 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:57.260027    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 18:54:57.260216 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:57.260082    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv" podUID="98a5a772-0770-40c9-8a01-5b991fee55d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 18:54:58.247977 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:58.243330    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 18:54:58.247977 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:58.243390    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv" podUID="98a5a772-0770-40c9-8a01-5b991fee55d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 18:54:59.250391 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:59.250355    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 18:54:59.251030 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:59.250991    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv" podUID="98a5a772-0770-40c9-8a01-5b991fee55d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 18:54:59.660557 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:54:59.658552    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:00.256854 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:00.256817    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 18:55:00.257894 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:00.257863    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv" podUID="98a5a772-0770-40c9-8a01-5b991fee55d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 18:55:01.246977 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:01.243357    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 18:55:01.246977 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:01.243419    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv" podUID="98a5a772-0770-40c9-8a01-5b991fee55d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 18:55:02.239515 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:02.239421    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 18:55:02.239899 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:02.239504    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv" podUID="98a5a772-0770-40c9-8a01-5b991fee55d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 18:55:03.244322 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:03.244291    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:55:03.244794 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:03.244654    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:55:03.245410 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:03.245392    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 18:55:03.437372 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:03.437344    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr"
Oct 30 18:55:04.734854 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:04.734805    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:08.203927 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:55:08.203890    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:55:08.483742 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.483656    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"]
Oct 30 18:55:08.488051 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.488028    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.522164 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.522133    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-workload-certs\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.522164 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.522167    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istiod-ca-cert\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.522313 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.522286    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-data\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.522404 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.522389    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-workload-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.522440 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.522417    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvqqj\" (UniqueName: \"kubernetes.io/projected/1057f570-7bb3-4bfb-ab81-6696e3b792b2-kube-api-access-pvqqj\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.522516 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.522503    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-token\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.522566 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.522552    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-envoy\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.522637 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.522626    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-podinfo\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.522747 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.522734    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-credential-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.624012 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.623971    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-workload-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.624201 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.624028    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvqqj\" (UniqueName: \"kubernetes.io/projected/1057f570-7bb3-4bfb-ab81-6696e3b792b2-kube-api-access-pvqqj\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.624201 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.624082    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-token\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.624324 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.624229    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-envoy\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.624324 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.624299    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-podinfo\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.624437 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.624381    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-workload-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.624437 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.624395    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-credential-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.624512 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.624452    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-workload-certs\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.624512 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.624476    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istiod-ca-cert\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.624668 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.624537    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-data\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.624729 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.624681    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-credential-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.624860 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.624840    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-workload-certs\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.625117 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.624999    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-data\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.625163 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.625146    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istiod-ca-cert\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.626735 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.626716    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-envoy\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.627061 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.627041    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-podinfo\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.685232 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.685195    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-token\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.685374 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.685250    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvqqj\" (UniqueName: \"kubernetes.io/projected/1057f570-7bb3-4bfb-ab81-6696e3b792b2-kube-api-access-pvqqj\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:08.691166 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.691138    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"]
Oct 30 18:55:08.801562 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:08.801532    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:09.234843 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:09.234808    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"]
Oct 30 18:55:09.235274 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:55:09.235248    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1057f570_7bb3_4bfb_ab81_6696e3b792b2.slice/crio-a2b2d0bdebf6e5303bc6ad51c0882dd525c35638920fdc7a10ee1f3d0a85a402 WatchSource:0}: Error finding container a2b2d0bdebf6e5303bc6ad51c0882dd525c35638920fdc7a10ee1f3d0a85a402: Status 404 returned error can't find the container with id a2b2d0bdebf6e5303bc6ad51c0882dd525c35638920fdc7a10ee1f3d0a85a402
Oct 30 18:55:09.237861 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:09.237831    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:55:09.237933 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:09.237911    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:55:09.237998 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:09.237938    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:55:09.547793 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:09.546363    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd" event={"ID":"1057f570-7bb3-4bfb-ab81-6696e3b792b2","Type":"ContainerStarted","Data":"8df6b203ba45fcdbd4c34038515e9ce131e5a947a041b02ca5f093b78d708708"}
Oct 30 18:55:09.547793 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:09.546530    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd" event={"ID":"1057f570-7bb3-4bfb-ab81-6696e3b792b2","Type":"ContainerStarted","Data":"a2b2d0bdebf6e5303bc6ad51c0882dd525c35638920fdc7a10ee1f3d0a85a402"}
Oct 30 18:55:09.572936 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:09.572827    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:09.678050 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:09.677987    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd" podStartSLOduration=1.677922727 podStartE2EDuration="1.677922727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:09.674672151 +0000 UTC m=+1230.386420848" watchObservedRunningTime="2026-10-30 18:55:09.677922727 +0000 UTC m=+1230.389671425"
Oct 30 18:55:09.802157 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:09.802075    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:10.804287 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:10.804260    5820 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 18:55:10.804639 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:10.804307    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd" podUID="1057f570-7bb3-4bfb-ab81-6696e3b792b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 18:55:11.802675 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:11.802641    5820 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 18:55:11.802831 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:11.802703    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd" podUID="1057f570-7bb3-4bfb-ab81-6696e3b792b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 18:55:12.801832 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:12.801795    5820 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 18:55:12.802293 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:12.801859    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd" podUID="1057f570-7bb3-4bfb-ab81-6696e3b792b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 18:55:13.801818 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:13.801782    5820 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 18:55:13.802147 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:13.801838    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd" podUID="1057f570-7bb3-4bfb-ab81-6696e3b792b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 18:55:14.160881 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:14.160792    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"]
Oct 30 18:55:14.165942 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:14.165921    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"
Oct 30 18:55:14.216896 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:14.216859    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-maas-api-tls\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv\" (UID: \"fa7430bc-1b59-49c2-b93d-d1c5e40d3152\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"
Oct 30 18:55:14.217085 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:14.217039    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-openshift-service-ca\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv\" (UID: \"fa7430bc-1b59-49c2-b93d-d1c5e40d3152\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"
Oct 30 18:55:14.217172 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:14.217149    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-metrics-certs\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv\" (UID: \"fa7430bc-1b59-49c2-b93d-d1c5e40d3152\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"
Oct 30 18:55:14.217247 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:14.217231    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hj92\" (UniqueName: \"kubernetes.io/projected/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-kube-api-access-9hj92\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv\" (UID: \"fa7430bc-1b59-49c2-b93d-d1c5e40d3152\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"
Oct 30 18:55:14.219215 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:14.219193    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"]
Oct 30 18:55:14.264507 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:14.264476    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-66baf45c-serving-cert\""
Oct 30 18:55:14.319105 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:14.319076    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-maas-api-tls\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv\" (UID: \"fa7430bc-1b59-49c2-b93d-d1c5e40d3152\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"
Oct 30 18:55:14.319310 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:14.319124    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-openshift-service-ca\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv\" (UID: \"fa7430bc-1b59-49c2-b93d-d1c5e40d3152\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"
Oct 30 18:55:14.319310 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:14.319159    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-metrics-certs\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv\" (UID: \"fa7430bc-1b59-49c2-b93d-d1c5e40d3152\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"
Oct 30 18:55:14.319310 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:14.319195    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hj92\" (UniqueName: \"kubernetes.io/projected/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-kube-api-access-9hj92\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv\" (UID: \"fa7430bc-1b59-49c2-b93d-d1c5e40d3152\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"
Oct 30 18:55:14.319310 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:55:14.319245    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-serving-cert: secret "maas-api-e2e-mt-66baf45c-serving-cert" not found
Oct 30 18:55:14.319524 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:55:14.319346    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-maas-api-tls podName:fa7430bc-1b59-49c2-b93d-d1c5e40d3152 nodeName:}" failed. No retries permitted until 2026-10-30 18:55:14.819326064 +0000 UTC m=+1235.531074752 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-maas-api-tls") pod "maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv" (UID: "fa7430bc-1b59-49c2-b93d-d1c5e40d3152") : secret "maas-api-e2e-mt-66baf45c-serving-cert" not found
Oct 30 18:55:14.320031 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:14.320008    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-openshift-service-ca\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv\" (UID: \"fa7430bc-1b59-49c2-b93d-d1c5e40d3152\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"
Oct 30 18:55:14.321851 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:14.321832    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-metrics-certs\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv\" (UID: \"fa7430bc-1b59-49c2-b93d-d1c5e40d3152\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"
Oct 30 18:55:14.382108 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:14.382076    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hj92\" (UniqueName: \"kubernetes.io/projected/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-kube-api-access-9hj92\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv\" (UID: \"fa7430bc-1b59-49c2-b93d-d1c5e40d3152\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"
Oct 30 18:55:14.383692 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:14.383661    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:14.607101 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:14.607067    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:14.784827 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:14.784794    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:14.801980 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:14.801937    5820 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 18:55:14.802119 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:14.801993    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd" podUID="1057f570-7bb3-4bfb-ab81-6696e3b792b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 18:55:14.830821 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:14.830793    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-maas-api-tls\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv\" (UID: \"fa7430bc-1b59-49c2-b93d-d1c5e40d3152\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"
Oct 30 18:55:14.833306 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:14.833285    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-maas-api-tls\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv\" (UID: \"fa7430bc-1b59-49c2-b93d-d1c5e40d3152\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"
Oct 30 18:55:15.081101 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:15.081070    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"
Oct 30 18:55:15.478383 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:15.478354    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"]
Oct 30 18:55:15.480025 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:55:15.479995    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa7430bc_1b59_49c2_b93d_d1c5e40d3152.slice/crio-b6fb000c9c084f7407b01208dd72f835120bb2035a50665962fc694ab042ceda WatchSource:0}: Error finding container b6fb000c9c084f7407b01208dd72f835120bb2035a50665962fc694ab042ceda: Status 404 returned error can't find the container with id b6fb000c9c084f7407b01208dd72f835120bb2035a50665962fc694ab042ceda
Oct 30 18:55:15.481597 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:15.481572    5820 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:55:15.607388 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:15.607349    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv" event={"ID":"fa7430bc-1b59-49c2-b93d-d1c5e40d3152","Type":"ContainerStarted","Data":"b6fb000c9c084f7407b01208dd72f835120bb2035a50665962fc694ab042ceda"}
Oct 30 18:55:15.802289 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:15.802259    5820 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 18:55:15.802444 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:15.802335    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd" podUID="1057f570-7bb3-4bfb-ab81-6696e3b792b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 18:55:16.616265 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:16.616226    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv" event={"ID":"fa7430bc-1b59-49c2-b93d-d1c5e40d3152","Type":"ContainerStarted","Data":"b20b73fc87edd25577f3f22515696bf04a830f095a235978ab40db9310d21f43"}
Oct 30 18:55:16.616675 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:16.616311    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"
Oct 30 18:55:16.737286 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:16.737185    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv" podStartSLOduration=1.834445983 podStartE2EDuration="2.737172083s" totalImagesPullingTime="902.7261ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:16.733523573 +0000 UTC m=+1237.445272282" watchObservedRunningTime="2026-10-30 18:55:16.737172083 +0000 UTC m=+1237.448920811"
Oct 30 18:55:16.806302 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:16.806275    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:16.806760 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:16.806740    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:16.807461 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:16.807445    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 18:55:19.327809 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:19.327766    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:22.456075 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:22.456026    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt" podUID="e31dcc89-d59d-4532-bda4-f28eb3b91095" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:55:22.591107 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:22.591042    5820 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n" podUID="80773b82-666e-428b-aeec-361a6c9cd6ac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:55:22.649899 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:22.649868    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"
Oct 30 18:55:22.752904 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:22.752812    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk" podUID="7300a782-cf82-48ef-a0da-0429b80a3480" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:55:22.765052 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:22.765013    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n" podUID="80773b82-666e-428b-aeec-361a6c9cd6ac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 18:55:23.045747 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:23.045378    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-qxlbf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:55:23.045747 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:23.045435    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf" podUID="76921ad3-c8c7-4de8-b9ef-5198a8c94327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:55:24.574170 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:24.574069    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:29.433944 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:29.433904    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:31.956779 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:31.956732    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:34.267475 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:34.267439    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:38.204243 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:55:38.204206    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:55:39.504976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:39.502641    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:44.331079 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:44.331043    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:46.669856 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:46.669801    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:49.260532 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:49.260436    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:53.416645 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:53.416606    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:54.244182 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:54.244147    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:58.709975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:58.709345    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:59.391973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:55:59.378073    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:04.162027 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:04.161984    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:08.203462 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:56:08.203415    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:56:08.932886 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:08.932849    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:09.127092 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.127050    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:09.628279 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.628249    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"]
Oct 30 18:56:09.632499 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.632484    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:09.689664 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.689632    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-10236a-openshift-default-dockercfg-258pk\""
Oct 30 18:56:09.709900 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.709849    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istiod-ca-cert\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:09.710132 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.709910    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-workload-certs\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:09.710132 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.709980    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9z8m\" (UniqueName: \"kubernetes.io/projected/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-kube-api-access-r9z8m\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:09.710132 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.710011    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-credential-socket\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:09.710378 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.710165    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-workload-socket\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:09.710378 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.710214    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-envoy\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:09.710378 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.710281    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-token\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:09.710378 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.710327    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-data\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:09.710619 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.710396    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-podinfo\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:09.825975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.814533    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:09.927171 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.919855    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istiod-ca-cert\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:09.927171 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.919913    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-workload-certs\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:09.927171 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.921212    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istiod-ca-cert\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:09.959747 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.954629    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9z8m\" (UniqueName: \"kubernetes.io/projected/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-kube-api-access-r9z8m\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:09.959747 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.954683    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-credential-socket\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:09.959747 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.954839    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-workload-socket\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:09.959747 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.954885    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-envoy\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:09.959747 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.955011    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-token\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:09.959747 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.955062    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-data\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:09.959747 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.955137    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-podinfo\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:09.979990 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.976096    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-credential-socket\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:09.979990 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.976379    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-workload-certs\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:09.979990 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:09.979869    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-workload-socket\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:10.001389 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:10.001357    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-envoy\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:10.006554 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:10.001371    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-data\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:10.013070 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:10.013034    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"]
Oct 30 18:56:10.014213 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:10.014187    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-podinfo\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:10.032652 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:10.032608    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:10.072286 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:10.072021    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9z8m\" (UniqueName: \"kubernetes.io/projected/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-kube-api-access-r9z8m\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:10.073963 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:10.073307    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-token\") pod \"e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") " pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:10.247217 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:10.247127    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:10.686199 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:10.686165    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"]
Oct 30 18:56:10.689141 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:56:10.689114    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0ab86cb_79ad_43d5_ae1b_13b4e0bd1c97.slice/crio-ddaa1187cfba65ad5ce12cf3150955372765ddc8421564d9209f66eb30c11219 WatchSource:0}: Error finding container ddaa1187cfba65ad5ce12cf3150955372765ddc8421564d9209f66eb30c11219: Status 404 returned error can't find the container with id ddaa1187cfba65ad5ce12cf3150955372765ddc8421564d9209f66eb30c11219
Oct 30 18:56:10.691491 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:10.691457    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:56:10.691577 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:10.691525    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:56:10.691577 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:10.691556    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:56:11.168865 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:11.168830    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp" event={"ID":"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97","Type":"ContainerStarted","Data":"cb3a336f58660453f2b18cbcb5d6a31347bf0ac0d0bb845060361484785de024"}
Oct 30 18:56:11.169076 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:11.168917    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp" event={"ID":"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97","Type":"ContainerStarted","Data":"ddaa1187cfba65ad5ce12cf3150955372765ddc8421564d9209f66eb30c11219"}
Oct 30 18:56:11.247626 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:11.247593    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:11.253226 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:11.253202    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:11.289742 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:11.289688    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp" podStartSLOduration=2.289675093 podStartE2EDuration="2.289675093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:11.288239003 +0000 UTC m=+1291.999987718" watchObservedRunningTime="2026-10-30 18:56:11.289675093 +0000 UTC m=+1292.001423791"
Oct 30 18:56:12.175104 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:12.175070    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:12.176122 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:12.176088    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:56:13.894814 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:13.894780    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:14.232721 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:14.232634    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:14.415258 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:14.415224    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:14.590228 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:14.590199    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:17.485097 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:17.485057    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:17.683136 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:17.681087    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:18.867982 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:18.866871    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:23.637994 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:23.637938    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:28.461300 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:28.461266    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:30.770113 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:30.770076    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:31.936385 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:31.936325    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:33.365974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:33.360438    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:38.087564 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:38.087518    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:38.209564 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:56:38.209523    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:56:38.263550 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:38.263515    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:42.861977 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:42.861748    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:48.105433 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:48.105393    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:48.147863 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:48.147828    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"]
Oct 30 18:56:48.187976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:48.179788    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"]
Oct 30 18:56:48.187976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:48.179905    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"
Oct 30 18:56:48.233704 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:48.233670    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-10236a-serving-cert\""
Oct 30 18:56:48.282917 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:48.282877    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5drd\" (UniqueName: \"kubernetes.io/projected/0d3dbc80-0283-496b-a153-8d7ae01b7946-kube-api-access-d5drd\") pod \"maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4\" (UID: \"0d3dbc80-0283-496b-a153-8d7ae01b7946\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"
Oct 30 18:56:48.283253 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:48.283225    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3dbc80-0283-496b-a153-8d7ae01b7946-openshift-service-ca\") pod \"maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4\" (UID: \"0d3dbc80-0283-496b-a153-8d7ae01b7946\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"
Oct 30 18:56:48.283588 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:48.283566    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d3dbc80-0283-496b-a153-8d7ae01b7946-metrics-certs\") pod \"maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4\" (UID: \"0d3dbc80-0283-496b-a153-8d7ae01b7946\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"
Oct 30 18:56:48.283732 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:48.283716    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d3dbc80-0283-496b-a153-8d7ae01b7946-maas-api-tls\") pod \"maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4\" (UID: \"0d3dbc80-0283-496b-a153-8d7ae01b7946\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"
Oct 30 18:56:48.308224 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:48.308194    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:48.387241 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:48.387136    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d3dbc80-0283-496b-a153-8d7ae01b7946-metrics-certs\") pod \"maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4\" (UID: \"0d3dbc80-0283-496b-a153-8d7ae01b7946\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"
Oct 30 18:56:48.387511 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:48.387479    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d3dbc80-0283-496b-a153-8d7ae01b7946-maas-api-tls\") pod \"maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4\" (UID: \"0d3dbc80-0283-496b-a153-8d7ae01b7946\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"
Oct 30 18:56:48.387820 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:48.387799    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5drd\" (UniqueName: \"kubernetes.io/projected/0d3dbc80-0283-496b-a153-8d7ae01b7946-kube-api-access-d5drd\") pod \"maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4\" (UID: \"0d3dbc80-0283-496b-a153-8d7ae01b7946\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"
Oct 30 18:56:48.388057 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:48.388025    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3dbc80-0283-496b-a153-8d7ae01b7946-openshift-service-ca\") pod \"maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4\" (UID: \"0d3dbc80-0283-496b-a153-8d7ae01b7946\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"
Oct 30 18:56:48.389838 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:48.389812    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3dbc80-0283-496b-a153-8d7ae01b7946-openshift-service-ca\") pod \"maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4\" (UID: \"0d3dbc80-0283-496b-a153-8d7ae01b7946\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"
Oct 30 18:56:48.398511 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:48.398488    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d3dbc80-0283-496b-a153-8d7ae01b7946-metrics-certs\") pod \"maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4\" (UID: \"0d3dbc80-0283-496b-a153-8d7ae01b7946\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"
Oct 30 18:56:48.446008 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:48.445978    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d3dbc80-0283-496b-a153-8d7ae01b7946-maas-api-tls\") pod \"maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4\" (UID: \"0d3dbc80-0283-496b-a153-8d7ae01b7946\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"
Oct 30 18:56:48.467120 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:48.467087    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5drd\" (UniqueName: \"kubernetes.io/projected/0d3dbc80-0283-496b-a153-8d7ae01b7946-kube-api-access-d5drd\") pod \"maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4\" (UID: \"0d3dbc80-0283-496b-a153-8d7ae01b7946\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"
Oct 30 18:56:48.520219 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:48.520183    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"
Oct 30 18:56:48.564827 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:48.564794    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:48.975978 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:48.935033    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk" podUID="7300a782-cf82-48ef-a0da-0429b80a3480" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:56:49.552852 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:49.552820    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"]
Oct 30 18:56:49.922112 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:49.922069    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n" podUID="80773b82-666e-428b-aeec-361a6c9cd6ac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 18:56:50.198412 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:50.198313    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4" event={"ID":"0d3dbc80-0283-496b-a153-8d7ae01b7946","Type":"ContainerStarted","Data":"da9ec89cc8b82de3a055ac5d8b0fdfe7117b29dfc655007de9d48cb4d52040b6"}
Oct 30 18:56:51.206294 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:51.206253    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4" event={"ID":"0d3dbc80-0283-496b-a153-8d7ae01b7946","Type":"ContainerStarted","Data":"61b2c8d8e0ea6f551a779bf4e9582e6d85f6f4b65a097ddb787c52f3fe1d89e6"}
Oct 30 18:56:51.206680 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:51.206338    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"
Oct 30 18:56:51.341580 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:51.341523    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4" podStartSLOduration=3.371146954 podStartE2EDuration="4.341509878s" totalImagesPullingTime="970.362924ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:51.338749969 +0000 UTC m=+1332.050498667" watchObservedRunningTime="2026-10-30 18:56:51.341509878 +0000 UTC m=+1332.053258610"
Oct 30 18:56:53.054973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:53.054871    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:53.740680 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:53.740643    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:57.221171 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:57.221134    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"
Oct 30 18:56:57.910334 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:56:57.910304    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:02.746397 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:02.746358    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:04.092656 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.092622    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:04.619758 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.619725    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"]
Oct 30 18:57:04.624064 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.624048    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.677584 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.677547    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-85ab6a-openshift-default-dockercfg-fqjx8\""
Oct 30 18:57:04.735297 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.735250    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-envoy\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.735882 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.735856    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-workload-certs\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.823975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.801073    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-data\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.823975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.801120    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-workload-socket\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.823975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.801177    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2844bd5-87c5-4f3c-86d4-d4041c484355-istiod-ca-cert\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.823975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.801222    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-token\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.823975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.801357    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-credential-socket\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.823975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.801378    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvn5p\" (UniqueName: \"kubernetes.io/projected/d2844bd5-87c5-4f3c-86d4-d4041c484355-kube-api-access-bvn5p\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.823975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.801504    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-podinfo\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.825594 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.825556    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:04.927025 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.924651    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-podinfo\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.927025 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.924752    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-envoy\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.927025 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.924799    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-workload-certs\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.927025 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.924841    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-data\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.927025 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.924867    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-workload-socket\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.927025 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.924905    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2844bd5-87c5-4f3c-86d4-d4041c484355-istiod-ca-cert\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.927025 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.924962    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-token\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.927025 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.925065    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-credential-socket\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.927025 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.925091    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvn5p\" (UniqueName: \"kubernetes.io/projected/d2844bd5-87c5-4f3c-86d4-d4041c484355-kube-api-access-bvn5p\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.937859 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.937755    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-workload-socket\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.938324 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.938288    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-workload-certs\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.969983 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.938996    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-data\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.970279 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.945417    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-credential-socket\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:04.970418 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:04.946154    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2844bd5-87c5-4f3c-86d4-d4041c484355-istiod-ca-cert\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:05.018890 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:05.006753    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"]
Oct 30 18:57:05.036166 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:05.036117    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-podinfo\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:05.037408 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:05.037376    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvn5p\" (UniqueName: \"kubernetes.io/projected/d2844bd5-87c5-4f3c-86d4-d4041c484355-kube-api-access-bvn5p\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:05.039001 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:05.038937    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-envoy\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:05.043446 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:05.043419    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-token\") pod \"e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") " pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:05.063512 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:05.063486    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:05.237597 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:05.237514    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:05.671160 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:05.671139    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"]
Oct 30 18:57:05.672514 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:57:05.672483    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2844bd5_87c5_4f3c_86d4_d4041c484355.slice/crio-8fd7212cddc8345454b544be3467490236f5df8f24c0545755d9cd3e362fcf62 WatchSource:0}: Error finding container 8fd7212cddc8345454b544be3467490236f5df8f24c0545755d9cd3e362fcf62: Status 404 returned error can't find the container with id 8fd7212cddc8345454b544be3467490236f5df8f24c0545755d9cd3e362fcf62
Oct 30 18:57:05.674979 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:05.674915    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:57:05.675054 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:05.675009    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:57:05.675054 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:05.675039    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:57:06.311795 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:06.311756    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l" event={"ID":"d2844bd5-87c5-4f3c-86d4-d4041c484355","Type":"ContainerStarted","Data":"438254d3c23b54711498fc918c159a8bb50f7b6275335c54032d451aa907dced"}
Oct 30 18:57:06.312262 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:06.311809    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l" event={"ID":"d2844bd5-87c5-4f3c-86d4-d4041c484355","Type":"ContainerStarted","Data":"8fd7212cddc8345454b544be3467490236f5df8f24c0545755d9cd3e362fcf62"}
Oct 30 18:57:06.433407 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:06.433360    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l" podStartSLOduration=2.4333443519999998 podStartE2EDuration="2.433344352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:06.430984792 +0000 UTC m=+1347.142733493" watchObservedRunningTime="2026-10-30 18:57:06.433344352 +0000 UTC m=+1347.145093051"
Oct 30 18:57:07.238335 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:07.238300    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:07.243606 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:07.243584    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:07.317763 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:07.317728    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:07.318655 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:07.318637    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:57:07.561034 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:07.553651    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:08.205026 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:57:08.204985    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:57:09.297560 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:09.297508    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:09.492690 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:09.492658    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:09.685407 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:09.685369    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:10.989433 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:10.989390    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"]
Oct 30 18:57:10.993867 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:10.993845    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.026962 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.026920    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gljr4\" (UniqueName: \"kubernetes.io/projected/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-kube-api-access-gljr4\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.027099 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.027020    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.027099 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.027067    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.027195 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.027106    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.027238 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.027216    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.027298 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.027249    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.027298 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.027292    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.027424 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.027400    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.027519 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.027502    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.038594 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.038569    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"]
Oct 30 18:57:11.042729 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.042711    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.095743 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.095703    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"]
Oct 30 18:57:11.128719 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.128684    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.128914 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.128738    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.128914 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.128828    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.128914 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.128877    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.129605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.128927    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.129605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.128971    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.129605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.129003    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gljr4\" (UniqueName: \"kubernetes.io/projected/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-kube-api-access-gljr4\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.129605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.129041    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.129605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.129094    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.129605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.129145    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.129605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.129177    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.129605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.129265    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.129605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.129287    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.129605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.129353    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.129605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.129382    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8q7b\" (UniqueName: \"kubernetes.io/projected/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-kube-api-access-f8q7b\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.129605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.129417    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.129605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.129458    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.129605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.129520    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.129605 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.129543    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.130318 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.129840    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.130318 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.130085    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.130318 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.130301    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.131229 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.131203    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.132172 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.131886    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.133641 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.133619    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.152999 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.148552    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"]
Oct 30 18:57:11.192307 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.192271    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gljr4\" (UniqueName: \"kubernetes.io/projected/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-kube-api-access-gljr4\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.193707 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.193686    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.232621 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.232587    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.232811 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.232637    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8q7b\" (UniqueName: \"kubernetes.io/projected/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-kube-api-access-f8q7b\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.232887 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.232807    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.232994 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.232886    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.232994 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.232911    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.233119 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.233081    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.233190 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.233171    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.233246 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.233202    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.233325 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.233302    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.233400 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.233307    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.233461 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.233409    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.233821 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.233799    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.234137 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.234111    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.234398 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.234369    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.235555 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.235535    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.236281 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.236261    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.293558 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.293498    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8q7b\" (UniqueName: \"kubernetes.io/projected/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-kube-api-access-f8q7b\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.294043 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.294020    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.309554 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.309535    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:11.356189 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.356160    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:11.741600 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.741572    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"]
Oct 30 18:57:11.742583 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:57:11.742549    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcef9b8a_2a3a_4e51_875d_131d764e1ed0.slice/crio-65fd26b72935535f0ebcd91a2085b7a7f9a6871f30ab7da39b2a07c2fd64ae5c WatchSource:0}: Error finding container 65fd26b72935535f0ebcd91a2085b7a7f9a6871f30ab7da39b2a07c2fd64ae5c: Status 404 returned error can't find the container with id 65fd26b72935535f0ebcd91a2085b7a7f9a6871f30ab7da39b2a07c2fd64ae5c
Oct 30 18:57:11.745285 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.745250    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:57:11.745378 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.745317    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:57:11.745378 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.745344    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:57:11.796904 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.796882    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"]
Oct 30 18:57:11.799241 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:57:11.799204    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod562448ab_2cf6_4fe5_b2f0_0c227a4ef70a.slice/crio-6294b73cc2dc8f2bc4c7ebf29de61536229c87e80c4bd74fa4b909b883c4e39c WatchSource:0}: Error finding container 6294b73cc2dc8f2bc4c7ebf29de61536229c87e80c4bd74fa4b909b883c4e39c: Status 404 returned error can't find the container with id 6294b73cc2dc8f2bc4c7ebf29de61536229c87e80c4bd74fa4b909b883c4e39c
Oct 30 18:57:11.802319 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.802289    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:57:11.802389 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.802350    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:57:11.802389 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:11.802380    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:57:12.360005 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:12.359623    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms" event={"ID":"fcef9b8a-2a3a-4e51-875d-131d764e1ed0","Type":"ContainerStarted","Data":"751f68cb8d9b7adf06705351fa4d4a2eaaaaac525c08300a50bf84c93f0b0076"}
Oct 30 18:57:12.360005 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:12.359701    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms" event={"ID":"fcef9b8a-2a3a-4e51-875d-131d764e1ed0","Type":"ContainerStarted","Data":"65fd26b72935535f0ebcd91a2085b7a7f9a6871f30ab7da39b2a07c2fd64ae5c"}
Oct 30 18:57:12.363188 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:12.363156    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8" event={"ID":"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a","Type":"ContainerStarted","Data":"dc2babf0a5be41fcba7e033b1f2208215296bf988cb598b0ca9ff6fa72579e1a"}
Oct 30 18:57:12.363431 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:12.363400    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8" event={"ID":"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a","Type":"ContainerStarted","Data":"6294b73cc2dc8f2bc4c7ebf29de61536229c87e80c4bd74fa4b909b883c4e39c"}
Oct 30 18:57:12.484175 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:12.484114    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms" podStartSLOduration=2.484094502 podStartE2EDuration="2.484094502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:12.478516197 +0000 UTC m=+1353.190264912" watchObservedRunningTime="2026-10-30 18:57:12.484094502 +0000 UTC m=+1353.195843205"
Oct 30 18:57:12.595657 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:12.595627    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:12.902976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:12.899623    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:13.320025 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:13.316062    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:13.360677 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:13.360632    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 18:57:13.370907 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:13.361290    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms" podUID="fcef9b8a-2a3a-4e51-875d-131d764e1ed0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 18:57:13.416149 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:13.416060    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:13.476086 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:13.476045    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Oct 30 18:57:13.476293 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:13.476099    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8" podUID="562448ab-2cf6-4fe5-b2f0-0c227a4ef70a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Oct 30 18:57:14.310847 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:14.310807    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 18:57:14.311026 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:14.310887    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms" podUID="fcef9b8a-2a3a-4e51-875d-131d764e1ed0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 18:57:14.364975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:14.364543    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Oct 30 18:57:14.364975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:14.364616    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8" podUID="562448ab-2cf6-4fe5-b2f0-0c227a4ef70a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Oct 30 18:57:15.310413 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:15.310377    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 18:57:15.310596 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:15.310444    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms" podUID="fcef9b8a-2a3a-4e51-875d-131d764e1ed0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 18:57:15.362369 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:15.362330    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Oct 30 18:57:15.362648 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:15.362620    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8" podUID="562448ab-2cf6-4fe5-b2f0-0c227a4ef70a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Oct 30 18:57:16.310200 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:16.310158    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 18:57:16.310612 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:16.310225    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms" podUID="fcef9b8a-2a3a-4e51-875d-131d764e1ed0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 18:57:16.358976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:16.358351    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Oct 30 18:57:16.358976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:16.358418    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8" podUID="562448ab-2cf6-4fe5-b2f0-0c227a4ef70a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Oct 30 18:57:17.310798 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:17.310719    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 18:57:17.310798 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:17.310770    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms" podUID="fcef9b8a-2a3a-4e51-875d-131d764e1ed0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 18:57:17.357380 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:17.357341    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Oct 30 18:57:17.357671 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:17.357641    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8" podUID="562448ab-2cf6-4fe5-b2f0-0c227a4ef70a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Oct 30 18:57:17.841974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:17.840278    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:18.310682 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:18.310650    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 18:57:18.310891 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:18.310718    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms" podUID="fcef9b8a-2a3a-4e51-875d-131d764e1ed0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 18:57:18.360368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:18.360327    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Oct 30 18:57:18.360659 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:18.360629    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8" podUID="562448ab-2cf6-4fe5-b2f0-0c227a4ef70a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Oct 30 18:57:19.358389 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:19.358350    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Oct 30 18:57:19.359009 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:19.358972    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8" podUID="562448ab-2cf6-4fe5-b2f0-0c227a4ef70a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Oct 30 18:57:19.730011 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:19.729897    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:19.730962 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:19.730922    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:19.733292 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:19.733274    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 18:57:19.902812 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:19.902748    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8" podStartSLOduration=9.902728769 podStartE2EDuration="9.902728769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:12.670769407 +0000 UTC m=+1353.382518105" watchObservedRunningTime="2026-10-30 18:57:19.902728769 +0000 UTC m=+1360.614477468"
Oct 30 18:57:21.190493 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:21.190459    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:21.191781 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:21.191750    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:21.192746 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:21.192728    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 18:57:22.818291 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:22.818249    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:27.593973 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:27.592024    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:28.217132 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:28.217084    5820 patch_prober.go:29] interesting pod/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:57:28.217316 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:28.217156    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp" podUID="d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:57:30.674936 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:30.674906    5820 generic.go:356] "Generic (PLEG): container finished" podID="d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" containerID="cb3a336f58660453f2b18cbcb5d6a31347bf0ac0d0bb845060361484785de024" exitCode=0
Oct 30 18:57:30.675427 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:30.674990    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp" event={"ID":"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97","Type":"ContainerDied","Data":"cb3a336f58660453f2b18cbcb5d6a31347bf0ac0d0bb845060361484785de024"}
Oct 30 18:57:30.675508 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:30.675463    5820 scope.go:122] "RemoveContainer" containerID="cb3a336f58660453f2b18cbcb5d6a31347bf0ac0d0bb845060361484785de024"
Oct 30 18:57:30.677221 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:30.677187    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:57:30.677309 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:30.677271    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:57:30.677349 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:30.677317    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:57:31.883735 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:31.883696    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp" event={"ID":"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97","Type":"ContainerStarted","Data":"fe09d43c20db9515a706adcc41f5f15e9b5f9de9fdbf16b99b5649743fff832b"}
Oct 30 18:57:32.259255 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:32.259132    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:57:32.259429 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:32.259363    5820 patch_prober.go:29] interesting pod/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 18:57:32.259492 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:32.259423    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp" podUID="d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 18:57:32.455261 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:32.455223    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:33.259161 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:33.259127    5820 patch_prober.go:29] interesting pod/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 18:57:33.259681 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:33.259200    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp" podUID="d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 18:57:34.259138 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:34.259100    5820 patch_prober.go:29] interesting pod/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 18:57:34.259371 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:34.259160    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp" podUID="d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 18:57:35.250792 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:35.250338    5820 patch_prober.go:29] interesting pod/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 18:57:35.250792 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:35.250396    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp" podUID="d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 18:57:35.840657 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:35.840622    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:36.259145 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:36.259056    5820 patch_prober.go:29] interesting pod/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 18:57:36.259145 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:36.259119    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp" podUID="d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 18:57:36.555534 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:36.555498    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:37.417812 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:37.417774    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:37.472266 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:37.472237    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:57:37.472779 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:37.472752    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:57:37.474086 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:37.474066    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 18:57:37.623476 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:37.623442    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:37.812065 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:37.812033    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:38.203652 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:57:38.203562    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:57:39.181599 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.181564    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:39.692910 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.692882    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"]
Oct 30 18:57:39.697322 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.697301    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.763970 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.758426    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-workload-socket\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.763970 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.758507    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-workload-certs\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.763970 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.758685    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6wv4\" (UniqueName: \"kubernetes.io/projected/a98db9fc-3545-4eca-a2b5-964583229a01-kube-api-access-v6wv4\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.763970 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.758735    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-istio-envoy\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.763970 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.758817    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a98db9fc-3545-4eca-a2b5-964583229a01-istio-podinfo\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.763970 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.758872    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a98db9fc-3545-4eca-a2b5-964583229a01-istiod-ca-cert\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.763970 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.758921    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a98db9fc-3545-4eca-a2b5-964583229a01-istio-token\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.763970 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.759223    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-credential-socket\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.763970 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.759307    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-istio-data\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.932495 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.896090    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-83d5bd-openshift-default-dockercfg-rd4wd\""
Oct 30 18:57:39.932495 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.921700    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:39.968265 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.968181    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-credential-socket\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.968568 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.968544    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-istio-data\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.968819 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.968791    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-workload-socket\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.969080 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.969063    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-workload-certs\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.969359 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.969339    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6wv4\" (UniqueName: \"kubernetes.io/projected/a98db9fc-3545-4eca-a2b5-964583229a01-kube-api-access-v6wv4\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.969627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.969530    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-istio-envoy\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.969790 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.969738    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a98db9fc-3545-4eca-a2b5-964583229a01-istio-podinfo\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.969935 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.969920    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a98db9fc-3545-4eca-a2b5-964583229a01-istiod-ca-cert\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.970113 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.970095    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a98db9fc-3545-4eca-a2b5-964583229a01-istio-token\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.970739 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.970713    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-workload-certs\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.992630 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.972398    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-istio-data\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.992930 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.972607    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-workload-socket\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.993596 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.984037    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a98db9fc-3545-4eca-a2b5-964583229a01-istiod-ca-cert\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:39.993753 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:39.989418    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-credential-socket\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:40.057894 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:40.056696    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a98db9fc-3545-4eca-a2b5-964583229a01-istio-podinfo\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:40.133756 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:40.133720    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-istio-envoy\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:40.147383 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:40.147252    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"]
Oct 30 18:57:40.168246 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:40.168175    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a98db9fc-3545-4eca-a2b5-964583229a01-istio-token\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:40.199325 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:40.199272    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6wv4\" (UniqueName: \"kubernetes.io/projected/a98db9fc-3545-4eca-a2b5-964583229a01-kube-api-access-v6wv4\") pod \"e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") " pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:40.321677 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:40.321563    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:40.362335 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:40.362293    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:40.761417 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:40.761390    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"]
Oct 30 18:57:40.761590 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:57:40.761566    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda98db9fc_3545_4eca_a2b5_964583229a01.slice/crio-4a3495fcb90240952601482bb0d34fc801dd348c6faf85c66e3e99c0f4061e71 WatchSource:0}: Error finding container 4a3495fcb90240952601482bb0d34fc801dd348c6faf85c66e3e99c0f4061e71: Status 404 returned error can't find the container with id 4a3495fcb90240952601482bb0d34fc801dd348c6faf85c66e3e99c0f4061e71
Oct 30 18:57:40.764503 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:40.764472    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:57:40.764579 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:40.764538    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:57:40.764579 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:40.764565    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:57:41.075052 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:41.075013    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn" event={"ID":"a98db9fc-3545-4eca-a2b5-964583229a01","Type":"ContainerStarted","Data":"b4c5572aa1125e96062decafb7d2efc601feb6bcce7e1b92f90675408f277fbd"}
Oct 30 18:57:41.075220 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:41.075073    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn" event={"ID":"a98db9fc-3545-4eca-a2b5-964583229a01","Type":"ContainerStarted","Data":"4a3495fcb90240952601482bb0d34fc801dd348c6faf85c66e3e99c0f4061e71"}
Oct 30 18:57:41.196451 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:41.196402    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn" podStartSLOduration=2.196388656 podStartE2EDuration="2.196388656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:41.195046854 +0000 UTC m=+1381.906795552" watchObservedRunningTime="2026-10-30 18:57:41.196388656 +0000 UTC m=+1381.908137353"
Oct 30 18:57:41.330022 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:41.323390    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:41.400978 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:41.400297    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:41.630133 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:41.630047    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:41.865816 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:41.865778    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:42.082020 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:42.081991    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:42.082944 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:42.082927    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:57:42.734015 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:42.728295    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:45.470166 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:45.470121    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:45.668704 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:45.662900    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:45.865208 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:45.864508    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:46.054816 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:46.054782    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r"]
Oct 30 18:57:46.088069 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:46.083542    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:46.090318 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:46.090290    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r"
Oct 30 18:57:46.171926 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:46.171841    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-470093\""
Oct 30 18:57:46.172855 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:46.172830    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-470093-dockercfg-qghbs\""
Oct 30 18:57:46.185617 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:46.185051    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj28b\" (UniqueName: \"kubernetes.io/projected/7454f83e-8a9e-4fcf-8468-6cd18c64a8b8-kube-api-access-hj28b\") pod \"payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r\" (UID: \"7454f83e-8a9e-4fcf-8468-6cd18c64a8b8\") " pod="openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r"
Oct 30 18:57:46.185617 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:46.185120    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7454f83e-8a9e-4fcf-8468-6cd18c64a8b8-plugins-config-volume\") pod \"payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r\" (UID: \"7454f83e-8a9e-4fcf-8468-6cd18c64a8b8\") " pod="openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r"
Oct 30 18:57:46.192112 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:46.192086    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r"]
Oct 30 18:57:46.286369 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:46.286325    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj28b\" (UniqueName: \"kubernetes.io/projected/7454f83e-8a9e-4fcf-8468-6cd18c64a8b8-kube-api-access-hj28b\") pod \"payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r\" (UID: \"7454f83e-8a9e-4fcf-8468-6cd18c64a8b8\") " pod="openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r"
Oct 30 18:57:46.286565 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:46.286432    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7454f83e-8a9e-4fcf-8468-6cd18c64a8b8-plugins-config-volume\") pod \"payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r\" (UID: \"7454f83e-8a9e-4fcf-8468-6cd18c64a8b8\") " pod="openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r"
Oct 30 18:57:46.287204 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:46.287184    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7454f83e-8a9e-4fcf-8468-6cd18c64a8b8-plugins-config-volume\") pod \"payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r\" (UID: \"7454f83e-8a9e-4fcf-8468-6cd18c64a8b8\") " pod="openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r"
Oct 30 18:57:46.347123 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:46.347093    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj28b\" (UniqueName: \"kubernetes.io/projected/7454f83e-8a9e-4fcf-8468-6cd18c64a8b8-kube-api-access-hj28b\") pod \"payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r\" (UID: \"7454f83e-8a9e-4fcf-8468-6cd18c64a8b8\") " pod="openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r"
Oct 30 18:57:46.535382 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:46.535304    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r"
Oct 30 18:57:46.912962 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:46.912922    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r"]
Oct 30 18:57:46.914085 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:57:46.914060    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7454f83e_8a9e_4fcf_8468_6cd18c64a8b8.slice/crio-d97540625d74d6ea20e6d99f0e0c10bf0c181a8c2dfd629e1c26f9c257d8409e WatchSource:0}: Error finding container d97540625d74d6ea20e6d99f0e0c10bf0c181a8c2dfd629e1c26f9c257d8409e: Status 404 returned error can't find the container with id d97540625d74d6ea20e6d99f0e0c10bf0c181a8c2dfd629e1c26f9c257d8409e
Oct 30 18:57:47.226920 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:47.226845    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r" event={"ID":"7454f83e-8a9e-4fcf-8468-6cd18c64a8b8","Type":"ContainerStarted","Data":"58addef4eef7e5b1c52c2bd6b8be99dd153528b97eae32cd0ffd3cec10c0b9ba"}
Oct 30 18:57:47.226920 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:47.226892    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r" event={"ID":"7454f83e-8a9e-4fcf-8468-6cd18c64a8b8","Type":"ContainerStarted","Data":"d97540625d74d6ea20e6d99f0e0c10bf0c181a8c2dfd629e1c26f9c257d8409e"}
Oct 30 18:57:47.227142 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:47.227041    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r"
Oct 30 18:57:47.349257 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:47.349208    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r" podStartSLOduration=2.3491949979999998 podStartE2EDuration="2.349194998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:47.346185348 +0000 UTC m=+1388.057934047" watchObservedRunningTime="2026-10-30 18:57:47.349194998 +0000 UTC m=+1388.060943694"
Oct 30 18:57:48.052935 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:48.052894    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:48.264976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:48.261328    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:48.434434 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:48.434349    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:53.043559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:53.043524    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:53.312030 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:53.299671    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:54.114608 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:54.114562    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:57.940085 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:57.940034    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:58.289987 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:57:58.289881    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r"
Oct 30 18:58:02.134569 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:02.134467    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:02.314009 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:02.313831    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:03.118975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:03.107015    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:03.436036 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:03.435054    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt" podUID="e31dcc89-d59d-4532-bda4-f28eb3b91095" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:58:03.684536 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:03.684490    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw" podUID="1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:58:04.038098 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:04.038046    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b" podUID="1cae2237-0916-415a-9059-b94e0d950b0a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:58:04.439146 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:04.439086    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt" podUID="e31dcc89-d59d-4532-bda4-f28eb3b91095" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:58:04.440863 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:04.440840    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:58:04.769255 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:04.769153    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n" podUID="80773b82-666e-428b-aeec-361a6c9cd6ac" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:58:04.824110 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:04.824065    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk" podUID="7300a782-cf82-48ef-a0da-0429b80a3480" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:58:04.938090 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:04.938055    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 18:58:08.161438 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:08.161384    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:08.293698 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:58:08.293649    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:58:12.614001 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:12.613941    5820 generic.go:356] "Generic (PLEG): container finished" podID="d2844bd5-87c5-4f3c-86d4-d4041c484355" containerID="438254d3c23b54711498fc918c159a8bb50f7b6275335c54032d451aa907dced" exitCode=0
Oct 30 18:58:12.614493 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:12.614015    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l" event={"ID":"d2844bd5-87c5-4f3c-86d4-d4041c484355","Type":"ContainerDied","Data":"438254d3c23b54711498fc918c159a8bb50f7b6275335c54032d451aa907dced"}
Oct 30 18:58:12.614493 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:12.614373    5820 scope.go:122] "RemoveContainer" containerID="438254d3c23b54711498fc918c159a8bb50f7b6275335c54032d451aa907dced"
Oct 30 18:58:12.615777 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:12.615746    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:58:12.615887 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:12.615829    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:58:12.615887 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:12.615879    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:58:13.108489 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:13.108454    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:13.366886 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:13.366801    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:13.571201 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:13.571171    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:13.623226 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:13.623150    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l" event={"ID":"d2844bd5-87c5-4f3c-86d4-d4041c484355","Type":"ContainerStarted","Data":"f6f79e29a8155849464356ea5327e87a2acd0e7c4ab4615086112f7cafda5e5d"}
Oct 30 18:58:14.254974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:14.246319    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:58:14.254974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:14.246510    5820 patch_prober.go:29] interesting pod/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 18:58:14.254974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:14.246574    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l" podUID="d2844bd5-87c5-4f3c-86d4-d4041c484355" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 18:58:15.253974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:15.243854    5820 patch_prober.go:29] interesting pod/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 18:58:15.253974 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:15.243914    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l" podUID="d2844bd5-87c5-4f3c-86d4-d4041c484355" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 18:58:16.248972 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:16.243362    5820 patch_prober.go:29] interesting pod/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 18:58:16.248972 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:16.243436    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l" podUID="d2844bd5-87c5-4f3c-86d4-d4041c484355" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 18:58:17.259139 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:17.259050    5820 patch_prober.go:29] interesting pod/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 18:58:17.259139 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:17.259120    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l" podUID="d2844bd5-87c5-4f3c-86d4-d4041c484355" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 18:58:18.251975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:18.242299    5820 patch_prober.go:29] interesting pod/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 18:58:18.251975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:18.242359    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l" podUID="d2844bd5-87c5-4f3c-86d4-d4041c484355" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 18:58:18.356854 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:18.356817    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:18.977121 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:18.977085    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:19.167334 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:19.167303    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:19.243219 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:19.243138    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:58:19.243512 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:19.243494    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:58:19.244292 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:19.244276    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 18:58:23.784514 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:23.776615    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:28.766774 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:28.766743    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"]
Oct 30 18:58:28.770173 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:28.770151    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"
Oct 30 18:58:28.854035 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:28.854001    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc05bef2-24fa-4808-a429-8c6d450ed396-openshift-service-ca\") pod \"maas-api-e2e-api-b-470093-799d857fdc-fckcf\" (UID: \"bc05bef2-24fa-4808-a429-8c6d450ed396\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"
Oct 30 18:58:28.854222 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:28.854042    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ftd7\" (UniqueName: \"kubernetes.io/projected/bc05bef2-24fa-4808-a429-8c6d450ed396-kube-api-access-6ftd7\") pod \"maas-api-e2e-api-b-470093-799d857fdc-fckcf\" (UID: \"bc05bef2-24fa-4808-a429-8c6d450ed396\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"
Oct 30 18:58:28.854222 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:28.854137    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc05bef2-24fa-4808-a429-8c6d450ed396-metrics-certs\") pod \"maas-api-e2e-api-b-470093-799d857fdc-fckcf\" (UID: \"bc05bef2-24fa-4808-a429-8c6d450ed396\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"
Oct 30 18:58:28.854389 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:28.854372    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc05bef2-24fa-4808-a429-8c6d450ed396-maas-api-tls\") pod \"maas-api-e2e-api-b-470093-799d857fdc-fckcf\" (UID: \"bc05bef2-24fa-4808-a429-8c6d450ed396\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"
Oct 30 18:58:28.861438 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:28.861421    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-470093-serving-cert\""
Oct 30 18:58:28.897848 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:28.897822    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"]
Oct 30 18:58:28.955344 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:28.955315    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc05bef2-24fa-4808-a429-8c6d450ed396-openshift-service-ca\") pod \"maas-api-e2e-api-b-470093-799d857fdc-fckcf\" (UID: \"bc05bef2-24fa-4808-a429-8c6d450ed396\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"
Oct 30 18:58:28.955499 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:28.955351    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ftd7\" (UniqueName: \"kubernetes.io/projected/bc05bef2-24fa-4808-a429-8c6d450ed396-kube-api-access-6ftd7\") pod \"maas-api-e2e-api-b-470093-799d857fdc-fckcf\" (UID: \"bc05bef2-24fa-4808-a429-8c6d450ed396\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"
Oct 30 18:58:28.955499 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:28.955394    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc05bef2-24fa-4808-a429-8c6d450ed396-metrics-certs\") pod \"maas-api-e2e-api-b-470093-799d857fdc-fckcf\" (UID: \"bc05bef2-24fa-4808-a429-8c6d450ed396\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"
Oct 30 18:58:28.955599 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:28.955525    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc05bef2-24fa-4808-a429-8c6d450ed396-maas-api-tls\") pod \"maas-api-e2e-api-b-470093-799d857fdc-fckcf\" (UID: \"bc05bef2-24fa-4808-a429-8c6d450ed396\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"
Oct 30 18:58:28.955665 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:58:28.955644    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-470093-serving-cert: secret "maas-api-e2e-api-b-470093-serving-cert" not found
Oct 30 18:58:28.955736 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:58:28.955723    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc05bef2-24fa-4808-a429-8c6d450ed396-maas-api-tls podName:bc05bef2-24fa-4808-a429-8c6d450ed396 nodeName:}" failed. No retries permitted until 2026-10-30 18:58:29.45570051 +0000 UTC m=+1430.167449195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bc05bef2-24fa-4808-a429-8c6d450ed396-maas-api-tls") pod "maas-api-e2e-api-b-470093-799d857fdc-fckcf" (UID: "bc05bef2-24fa-4808-a429-8c6d450ed396") : secret "maas-api-e2e-api-b-470093-serving-cert" not found
Oct 30 18:58:28.956039 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:28.956021    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc05bef2-24fa-4808-a429-8c6d450ed396-openshift-service-ca\") pod \"maas-api-e2e-api-b-470093-799d857fdc-fckcf\" (UID: \"bc05bef2-24fa-4808-a429-8c6d450ed396\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"
Oct 30 18:58:28.958022 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:28.958007    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc05bef2-24fa-4808-a429-8c6d450ed396-metrics-certs\") pod \"maas-api-e2e-api-b-470093-799d857fdc-fckcf\" (UID: \"bc05bef2-24fa-4808-a429-8c6d450ed396\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"
Oct 30 18:58:29.021487 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:29.021420    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ftd7\" (UniqueName: \"kubernetes.io/projected/bc05bef2-24fa-4808-a429-8c6d450ed396-kube-api-access-6ftd7\") pod \"maas-api-e2e-api-b-470093-799d857fdc-fckcf\" (UID: \"bc05bef2-24fa-4808-a429-8c6d450ed396\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"
Oct 30 18:58:29.093230 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:29.092899    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:29.307294 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:29.307261    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:29.473187 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:29.473153    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc05bef2-24fa-4808-a429-8c6d450ed396-maas-api-tls\") pod \"maas-api-e2e-api-b-470093-799d857fdc-fckcf\" (UID: \"bc05bef2-24fa-4808-a429-8c6d450ed396\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"
Oct 30 18:58:29.475685 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:29.475664    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc05bef2-24fa-4808-a429-8c6d450ed396-maas-api-tls\") pod \"maas-api-e2e-api-b-470093-799d857fdc-fckcf\" (UID: \"bc05bef2-24fa-4808-a429-8c6d450ed396\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"
Oct 30 18:58:29.532403 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:29.532357    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:29.684811 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:29.684738    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"
Oct 30 18:58:30.074685 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:30.074654    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"]
Oct 30 18:58:30.075987 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:58:30.075934    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc05bef2_24fa_4808_a429_8c6d450ed396.slice/crio-e5d91cc41fa99ccd658e57767fcc3732e06fb0199792ac66ea49309142a8a0f9 WatchSource:0}: Error finding container e5d91cc41fa99ccd658e57767fcc3732e06fb0199792ac66ea49309142a8a0f9: Status 404 returned error can't find the container with id e5d91cc41fa99ccd658e57767fcc3732e06fb0199792ac66ea49309142a8a0f9
Oct 30 18:58:30.780534 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:30.780495    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf" event={"ID":"bc05bef2-24fa-4808-a429-8c6d450ed396","Type":"ContainerStarted","Data":"e5d91cc41fa99ccd658e57767fcc3732e06fb0199792ac66ea49309142a8a0f9"}
Oct 30 18:58:31.788866 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:31.788829    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf" event={"ID":"bc05bef2-24fa-4808-a429-8c6d450ed396","Type":"ContainerStarted","Data":"20e4b7e680b1efa699d373484487aeacf67cd9586249171c0060a18e580dba5f"}
Oct 30 18:58:31.789301 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:31.788976    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"
Oct 30 18:58:31.910559 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:31.910499    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf" podStartSLOduration=2.904663161 podStartE2EDuration="3.910479384s" totalImagesPullingTime="1.005816223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:31.908054442 +0000 UTC m=+1432.619803138" watchObservedRunningTime="2026-10-30 18:58:31.910479384 +0000 UTC m=+1432.622228083"
Oct 30 18:58:33.385832 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:33.385802    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr"]
Oct 30 18:58:33.725815 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:33.725736    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:33.966113 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:33.966071    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:34.579643 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:34.579608    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:34.755694 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:34.755661    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:34.942136 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:34.940677    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:37.823435 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:37.823406    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"
Oct 30 18:58:38.204121 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:58:38.204035    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:58:38.386644 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:38.386606    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr" podUID="68a37d9e-512a-4271-8916-6af9d7b69bdc" containerName="payload-pre-processing" containerID="cri-o://ad715e528bf2eb4c05fe3a092d7d05d271a2b6aff7c3b353b516401013c70702" gracePeriod=25
Oct 30 18:58:40.194972 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:40.193926    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:42.306025 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:42.305980    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:42.531643 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:42.531607    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:43.430713 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:43.430672    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr" podUID="68a37d9e-512a-4271-8916-6af9d7b69bdc" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 18:58:45.334033 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:45.329592    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:46.305967 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:46.305915    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:47.171694 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:47.171660    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:47.922902 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:47.922865    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:48.229057 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:48.228959    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:51.003971 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:51.001298    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:52.676975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:52.665660    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:52.984062 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:52.983973    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:53.278281 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:53.278168    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:53.495541 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:53.495069    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr" podUID="68a37d9e-512a-4271-8916-6af9d7b69bdc" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 18:58:56.947986 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:56.947938    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv"]
Oct 30 18:58:57.026678 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:57.026650    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv"]
Oct 30 18:58:57.027901 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:57.027201    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv"
Oct 30 18:58:57.054492 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:57.054076    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:57.090007 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:57.089153    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-132af1\""
Oct 30 18:58:57.126531 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:57.126501    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vhwl\" (UniqueName: \"kubernetes.io/projected/ee26eec2-d615-487b-b1b4-7ef2a0cc0516-kube-api-access-8vhwl\") pod \"payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv\" (UID: \"ee26eec2-d615-487b-b1b4-7ef2a0cc0516\") " pod="openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv"
Oct 30 18:58:57.126707 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:57.126675    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee26eec2-d615-487b-b1b4-7ef2a0cc0516-plugins-config-volume\") pod \"payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv\" (UID: \"ee26eec2-d615-487b-b1b4-7ef2a0cc0516\") " pod="openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv"
Oct 30 18:58:57.130900 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:57.130865    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-132af1-dockercfg-2tnhs\""
Oct 30 18:58:57.228397 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:57.228302    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vhwl\" (UniqueName: \"kubernetes.io/projected/ee26eec2-d615-487b-b1b4-7ef2a0cc0516-kube-api-access-8vhwl\") pod \"payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv\" (UID: \"ee26eec2-d615-487b-b1b4-7ef2a0cc0516\") " pod="openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv"
Oct 30 18:58:57.228551 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:57.228421    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee26eec2-d615-487b-b1b4-7ef2a0cc0516-plugins-config-volume\") pod \"payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv\" (UID: \"ee26eec2-d615-487b-b1b4-7ef2a0cc0516\") " pod="openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv"
Oct 30 18:58:57.229222 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:57.229200    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee26eec2-d615-487b-b1b4-7ef2a0cc0516-plugins-config-volume\") pod \"payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv\" (UID: \"ee26eec2-d615-487b-b1b4-7ef2a0cc0516\") " pod="openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv"
Oct 30 18:58:57.260005 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:57.257413    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:57.290122 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:57.290097    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vhwl\" (UniqueName: \"kubernetes.io/projected/ee26eec2-d615-487b-b1b4-7ef2a0cc0516-kube-api-access-8vhwl\") pod \"payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv\" (UID: \"ee26eec2-d615-487b-b1b4-7ef2a0cc0516\") " pod="openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv"
Oct 30 18:58:57.394256 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:57.394223    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv"
Oct 30 18:58:57.463981 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:57.463833    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:57.830074 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:57.830046    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv"]
Oct 30 18:58:57.830360 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:58:57.830339    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee26eec2_d615_487b_b1b4_7ef2a0cc0516.slice/crio-d4edca78e268801f5b083c61a4b8ca1dc63ee892a2e559d4e2197d571e3aad51 WatchSource:0}: Error finding container d4edca78e268801f5b083c61a4b8ca1dc63ee892a2e559d4e2197d571e3aad51: Status 404 returned error can't find the container with id d4edca78e268801f5b083c61a4b8ca1dc63ee892a2e559d4e2197d571e3aad51
Oct 30 18:58:58.127850 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:58.127766    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv" event={"ID":"ee26eec2-d615-487b-b1b4-7ef2a0cc0516","Type":"ContainerStarted","Data":"1d837f42f0600466f019399526c4a7b4ff9893a680e6c83eb7fce6e02ae0a1b8"}
Oct 30 18:58:58.127850 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:58.127821    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv" event={"ID":"ee26eec2-d615-487b-b1b4-7ef2a0cc0516","Type":"ContainerStarted","Data":"d4edca78e268801f5b083c61a4b8ca1dc63ee892a2e559d4e2197d571e3aad51"}
Oct 30 18:58:58.127850 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:58.127852    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv"
Oct 30 18:58:58.304881 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:58.304828    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv" podStartSLOduration=2.304815713 podStartE2EDuration="2.304815713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:58.255065246 +0000 UTC m=+1458.966813945" watchObservedRunningTime="2026-10-30 18:58:58.304815713 +0000 UTC m=+1459.016564434"
Oct 30 18:58:58.869572 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:58.869543    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr"
Oct 30 18:58:58.905183 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:58.905157    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/68a37d9e-512a-4271-8916-6af9d7b69bdc-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68a37d9e-512a-4271-8916-6af9d7b69bdc-plugins-config-volume\") pod \"68a37d9e-512a-4271-8916-6af9d7b69bdc\" (UID: \"68a37d9e-512a-4271-8916-6af9d7b69bdc\") "
Oct 30 18:58:58.905327 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:58.905264    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68a37d9e-512a-4271-8916-6af9d7b69bdc-kube-api-access-hrcpr\" (UniqueName: \"kubernetes.io/projected/68a37d9e-512a-4271-8916-6af9d7b69bdc-kube-api-access-hrcpr\") pod \"68a37d9e-512a-4271-8916-6af9d7b69bdc\" (UID: \"68a37d9e-512a-4271-8916-6af9d7b69bdc\") "
Oct 30 18:58:58.905516 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:58.905496    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68a37d9e-512a-4271-8916-6af9d7b69bdc-plugins-config-volume" pod "68a37d9e-512a-4271-8916-6af9d7b69bdc" (UID: "68a37d9e-512a-4271-8916-6af9d7b69bdc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:58.905775 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:58.905763    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68a37d9e-512a-4271-8916-6af9d7b69bdc-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:58:58.907566 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:58.907540    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68a37d9e-512a-4271-8916-6af9d7b69bdc-kube-api-access-hrcpr" pod "68a37d9e-512a-4271-8916-6af9d7b69bdc" (UID: "68a37d9e-512a-4271-8916-6af9d7b69bdc"). InnerVolumeSpecName "kube-api-access-hrcpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:59.007129 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:59.007044    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrcpr\" (UniqueName: \"kubernetes.io/projected/68a37d9e-512a-4271-8916-6af9d7b69bdc-kube-api-access-hrcpr\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:58:59.158977 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:59.154010    5820 generic.go:356] "Generic (PLEG): container finished" podID="68a37d9e-512a-4271-8916-6af9d7b69bdc" containerID="ad715e528bf2eb4c05fe3a092d7d05d271a2b6aff7c3b353b516401013c70702" exitCode=0
Oct 30 18:58:59.159472 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:59.159080    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr" event={"ID":"68a37d9e-512a-4271-8916-6af9d7b69bdc","Type":"ContainerDied","Data":"ad715e528bf2eb4c05fe3a092d7d05d271a2b6aff7c3b353b516401013c70702"}
Oct 30 18:58:59.159472 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:59.159119    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr"
Oct 30 18:58:59.159472 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:59.159166    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr" event={"ID":"68a37d9e-512a-4271-8916-6af9d7b69bdc","Type":"ContainerDied","Data":"89d2b45f26fdc0bf2d0307d40d5a7df9dceac3865b2c04a12659714bb2c87956"}
Oct 30 18:58:59.159472 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:59.159189    5820 scope.go:122] "RemoveContainer" containerID="ad715e528bf2eb4c05fe3a092d7d05d271a2b6aff7c3b353b516401013c70702"
Oct 30 18:58:59.170832 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:59.170814    5820 scope.go:122] "RemoveContainer" containerID="ad715e528bf2eb4c05fe3a092d7d05d271a2b6aff7c3b353b516401013c70702"
Oct 30 18:58:59.171135 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:58:59.171111    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad715e528bf2eb4c05fe3a092d7d05d271a2b6aff7c3b353b516401013c70702\": container with ID starting with ad715e528bf2eb4c05fe3a092d7d05d271a2b6aff7c3b353b516401013c70702 not found: ID does not exist" containerID="ad715e528bf2eb4c05fe3a092d7d05d271a2b6aff7c3b353b516401013c70702"
Oct 30 18:58:59.171218 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:59.171147    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad715e528bf2eb4c05fe3a092d7d05d271a2b6aff7c3b353b516401013c70702"} err="failed to get container status \"ad715e528bf2eb4c05fe3a092d7d05d271a2b6aff7c3b353b516401013c70702\": rpc error: code = NotFound desc = could not find container \"ad715e528bf2eb4c05fe3a092d7d05d271a2b6aff7c3b353b516401013c70702\": container with ID starting with ad715e528bf2eb4c05fe3a092d7d05d271a2b6aff7c3b353b516401013c70702 not found: ID does not exist"
Oct 30 18:58:59.333597 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:59.333568    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr"]
Oct 30 18:58:59.362075 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:59.362040    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-66baf45c-6794b75d75-hqdxr"]
Oct 30 18:58:59.442460 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:58:59.441857    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="68a37d9e-512a-4271-8916-6af9d7b69bdc" path="/var/lib/kubelet/pods/68a37d9e-512a-4271-8916-6af9d7b69bdc/volumes"
Oct 30 18:59:02.833065 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:02.824190    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:04.196898 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:04.196818    5820 generic.go:356] "Generic (PLEG): container finished" podID="a98db9fc-3545-4eca-a2b5-964583229a01" containerID="b4c5572aa1125e96062decafb7d2efc601feb6bcce7e1b92f90675408f277fbd" exitCode=0
Oct 30 18:59:04.197341 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:04.196898    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn" event={"ID":"a98db9fc-3545-4eca-a2b5-964583229a01","Type":"ContainerDied","Data":"b4c5572aa1125e96062decafb7d2efc601feb6bcce7e1b92f90675408f277fbd"}
Oct 30 18:59:04.197341 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:04.197331    5820 scope.go:122] "RemoveContainer" containerID="b4c5572aa1125e96062decafb7d2efc601feb6bcce7e1b92f90675408f277fbd"
Oct 30 18:59:04.198829 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:04.198792    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:59:04.198985 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:04.198882    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:59:04.198985 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:04.198933    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:59:05.205501 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:05.205466    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn" event={"ID":"a98db9fc-3545-4eca-a2b5-964583229a01","Type":"ContainerStarted","Data":"d42168379ca9c4657fbfecb07c367460e0ccc4dc1ffcc083a40c0bc38cd86ad2"}
Oct 30 18:59:05.322064 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:05.322038    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:59:05.322253 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:05.322231    5820 patch_prober.go:29] interesting pod/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 18:59:05.322305 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:05.322276    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn" podUID="a98db9fc-3545-4eca-a2b5-964583229a01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 18:59:06.321984 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:06.321911    5820 patch_prober.go:29] interesting pod/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 18:59:06.322475 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:06.322008    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn" podUID="a98db9fc-3545-4eca-a2b5-964583229a01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 18:59:07.322966 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:07.322925    5820 patch_prober.go:29] interesting pod/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 18:59:07.323360 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:07.323004    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn" podUID="a98db9fc-3545-4eca-a2b5-964583229a01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 18:59:08.205668 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:59:08.205629    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:59:08.306010 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:08.305976    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:08.322577 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:08.322537    5820 patch_prober.go:29] interesting pod/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 18:59:08.322896 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:08.322868    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn" podUID="a98db9fc-3545-4eca-a2b5-964583229a01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 18:59:09.165252 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:09.165216    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv"
Oct 30 18:59:09.322301 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:09.322262    5820 patch_prober.go:29] interesting pod/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 18:59:09.322460 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:09.322330    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn" podUID="a98db9fc-3545-4eca-a2b5-964583229a01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 18:59:10.328468 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:10.328435    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:59:10.329021 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:10.328923    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:59:10.329898 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:10.329875    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 18:59:13.729976 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:13.721266    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:18.849578 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:18.849539    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:19.262530 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:19.261386    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:24.480912 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:24.480867    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:26.199811 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:26.199769    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:29.576764 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:29.576723    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:32.101856 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:32.101818    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:32.341627 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:32.341583    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:34.169411 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:34.169365    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:34.215011 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:34.214977    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"]
Oct 30 18:59:34.215902 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:34.215858    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv" podUID="fa7430bc-1b59-49c2-b93d-d1c5e40d3152" containerName="maas-api" containerID="cri-o://b20b73fc87edd25577f3f22515696bf04a830f095a235978ab40db9310d21f43" gracePeriod=30
Oct 30 18:59:34.504064 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:34.490900    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:35.074869 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:35.074842    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"
Oct 30 18:59:35.170555 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:35.170520    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-kube-api-access-9hj92\" (UniqueName: \"kubernetes.io/projected/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-kube-api-access-9hj92\") pod \"fa7430bc-1b59-49c2-b93d-d1c5e40d3152\" (UID: \"fa7430bc-1b59-49c2-b93d-d1c5e40d3152\") "
Oct 30 18:59:35.171077 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:35.170624    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-metrics-certs\") pod \"fa7430bc-1b59-49c2-b93d-d1c5e40d3152\" (UID: \"fa7430bc-1b59-49c2-b93d-d1c5e40d3152\") "
Oct 30 18:59:35.171077 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:35.170787    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-openshift-service-ca\") pod \"fa7430bc-1b59-49c2-b93d-d1c5e40d3152\" (UID: \"fa7430bc-1b59-49c2-b93d-d1c5e40d3152\") "
Oct 30 18:59:35.171077 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:35.171052    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-maas-api-tls\") pod \"fa7430bc-1b59-49c2-b93d-d1c5e40d3152\" (UID: \"fa7430bc-1b59-49c2-b93d-d1c5e40d3152\") "
Oct 30 18:59:35.171394 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:35.171364    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-openshift-service-ca" pod "fa7430bc-1b59-49c2-b93d-d1c5e40d3152" (UID: "fa7430bc-1b59-49c2-b93d-d1c5e40d3152"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:59:35.171987 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:35.171963    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:59:35.173389 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:35.173296    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-kube-api-access-9hj92" pod "fa7430bc-1b59-49c2-b93d-d1c5e40d3152" (UID: "fa7430bc-1b59-49c2-b93d-d1c5e40d3152"). InnerVolumeSpecName "kube-api-access-9hj92". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:35.173809 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:35.173785    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-metrics-certs" pod "fa7430bc-1b59-49c2-b93d-d1c5e40d3152" (UID: "fa7430bc-1b59-49c2-b93d-d1c5e40d3152"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:59:35.176345 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:35.176316    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-maas-api-tls" pod "fa7430bc-1b59-49c2-b93d-d1c5e40d3152" (UID: "fa7430bc-1b59-49c2-b93d-d1c5e40d3152"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:59:35.274021 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:35.273922    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:59:35.274021 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:35.274008    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hj92\" (UniqueName: \"kubernetes.io/projected/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-kube-api-access-9hj92\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:59:35.274021 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:35.274024    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa7430bc-1b59-49c2-b93d-d1c5e40d3152-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:59:35.595883 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:35.595847    5820 generic.go:356] "Generic (PLEG): container finished" podID="fa7430bc-1b59-49c2-b93d-d1c5e40d3152" containerID="b20b73fc87edd25577f3f22515696bf04a830f095a235978ab40db9310d21f43" exitCode=0
Oct 30 18:59:35.596093 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:35.595895    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"
Oct 30 18:59:35.596093 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:35.595937    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv" event={"ID":"fa7430bc-1b59-49c2-b93d-d1c5e40d3152","Type":"ContainerDied","Data":"b20b73fc87edd25577f3f22515696bf04a830f095a235978ab40db9310d21f43"}
Oct 30 18:59:35.596093 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:35.596018    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv" event={"ID":"fa7430bc-1b59-49c2-b93d-d1c5e40d3152","Type":"ContainerDied","Data":"b6fb000c9c084f7407b01208dd72f835120bb2035a50665962fc694ab042ceda"}
Oct 30 18:59:35.596093 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:35.596048    5820 scope.go:122] "RemoveContainer" containerID="b20b73fc87edd25577f3f22515696bf04a830f095a235978ab40db9310d21f43"
Oct 30 18:59:35.606402 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:35.606379    5820 scope.go:122] "RemoveContainer" containerID="b20b73fc87edd25577f3f22515696bf04a830f095a235978ab40db9310d21f43"
Oct 30 18:59:35.606668 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:59:35.606649    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b20b73fc87edd25577f3f22515696bf04a830f095a235978ab40db9310d21f43\": container with ID starting with b20b73fc87edd25577f3f22515696bf04a830f095a235978ab40db9310d21f43 not found: ID does not exist" containerID="b20b73fc87edd25577f3f22515696bf04a830f095a235978ab40db9310d21f43"
Oct 30 18:59:35.606723 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:35.606679    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b20b73fc87edd25577f3f22515696bf04a830f095a235978ab40db9310d21f43"} err="failed to get container status \"b20b73fc87edd25577f3f22515696bf04a830f095a235978ab40db9310d21f43\": rpc error: code = NotFound desc = could not find container \"b20b73fc87edd25577f3f22515696bf04a830f095a235978ab40db9310d21f43\": container with ID starting with b20b73fc87edd25577f3f22515696bf04a830f095a235978ab40db9310d21f43 not found: ID does not exist"
Oct 30 18:59:35.767931 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:35.767901    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"]
Oct 30 18:59:35.816439 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:35.816362    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-j6vnv"]
Oct 30 18:59:36.125778 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:36.125700    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:36.456143 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:36.456060    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:36.659677 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:36.659641    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:37.352717 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:37.352680    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:37.438851 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:37.438819    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fa7430bc-1b59-49c2-b93d-d1c5e40d3152" path="/var/lib/kubelet/pods/fa7430bc-1b59-49c2-b93d-d1c5e40d3152/volumes"
Oct 30 18:59:37.937823 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:37.937786    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"]
Oct 30 18:59:37.948295 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:37.948257    5820 cpu_manager.go:414] "containerMap: removing container" podUID="fa7430bc-1b59-49c2-b93d-d1c5e40d3152" containerName="maas-api"
Oct 30 18:59:37.948557 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:37.948535    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa7430bc-1b59-49c2-b93d-d1c5e40d3152" containerName="maas-api"
Oct 30 18:59:37.948837 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:37.948809    5820 cpu_manager.go:414] "containerMap: removing container" podUID="68a37d9e-512a-4271-8916-6af9d7b69bdc" containerName="payload-pre-processing"
Oct 30 18:59:37.949006 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:37.948984    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68a37d9e-512a-4271-8916-6af9d7b69bdc" containerName="payload-pre-processing"
Oct 30 18:59:37.949368 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:37.949349    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa7430bc-1b59-49c2-b93d-d1c5e40d3152" containerName="maas-api"
Oct 30 18:59:37.949513 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:37.949500    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="68a37d9e-512a-4271-8916-6af9d7b69bdc" containerName="payload-pre-processing"
Oct 30 18:59:38.039752 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.021526    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.059480 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.059442    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"]
Oct 30 18:59:38.088777 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.078082    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:38.089373 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.078728    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-4ebcd2-openshift-default-dockercfg-2smq2\""
Oct 30 18:59:38.140830 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.140788    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-credential-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.155795 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.155762    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dece09c-bff0-403f-a991-b204dc1f0f4c-istiod-ca-cert\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.156229 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.156199    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-workload-certs\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.156687 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.156667    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx4kg\" (UniqueName: \"kubernetes.io/projected/5dece09c-bff0-403f-a991-b204dc1f0f4c-kube-api-access-dx4kg\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.156933 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.156918    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-podinfo\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.157159 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.157138    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-workload-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.157392 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.157370    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-token\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.157613 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.157594    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-envoy\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.157857 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.157837    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-data\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.221304 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:59:38.221211    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 18:59:38.279985 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.260146    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx4kg\" (UniqueName: \"kubernetes.io/projected/5dece09c-bff0-403f-a991-b204dc1f0f4c-kube-api-access-dx4kg\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.279985 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.260251    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-podinfo\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.279985 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.260293    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-workload-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.279985 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.260323    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-token\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.279985 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.260355    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-envoy\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.279985 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.260407    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-data\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.279985 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.260525    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-credential-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.279985 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.260557    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dece09c-bff0-403f-a991-b204dc1f0f4c-istiod-ca-cert\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.279985 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.260637    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-workload-certs\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.282061 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.281281    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-workload-certs\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.284929 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.283727    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-credential-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.285508 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.285445    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-data\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.286103 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.285998    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-workload-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.287326 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.287305    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dece09c-bff0-403f-a991-b204dc1f0f4c-istiod-ca-cert\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.295241 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.295208    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-envoy\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.302872 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.302803    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-podinfo\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.360789 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.360717    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-token\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.404493 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.397779    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx4kg\" (UniqueName: \"kubernetes.io/projected/5dece09c-bff0-403f-a991-b204dc1f0f4c-kube-api-access-dx4kg\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:38.404493 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.400734    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:38.696552 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:38.696247    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:39.261232 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:39.261197    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"]
Oct 30 18:59:39.289109 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:59:39.289082    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dece09c_bff0_403f_a991_b204dc1f0f4c.slice/crio-8d8293320641c82f0e85e47741587cf42a09c5d3703024fbe529afd59e8b739d WatchSource:0}: Error finding container 8d8293320641c82f0e85e47741587cf42a09c5d3703024fbe529afd59e8b739d: Status 404 returned error can't find the container with id 8d8293320641c82f0e85e47741587cf42a09c5d3703024fbe529afd59e8b739d
Oct 30 18:59:39.291771 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:39.291739    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:59:39.291830 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:39.291807    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:59:39.291870 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:39.291838    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 18:59:39.710069 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:39.710033    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms" event={"ID":"5dece09c-bff0-403f-a991-b204dc1f0f4c","Type":"ContainerStarted","Data":"44d724d76361f69e6e6b274c0cd909be0f1d819a288bc7adc57196990cfce6f9"}
Oct 30 18:59:39.710290 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:39.710091    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms" event={"ID":"5dece09c-bff0-403f-a991-b204dc1f0f4c","Type":"ContainerStarted","Data":"8d8293320641c82f0e85e47741587cf42a09c5d3703024fbe529afd59e8b739d"}
Oct 30 18:59:39.883029 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:39.882938    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms" podStartSLOduration=2.8829248339999998 podStartE2EDuration="2.882924834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:39.8338629 +0000 UTC m=+1500.545611597" watchObservedRunningTime="2026-10-30 18:59:39.882924834 +0000 UTC m=+1500.594673530"
Oct 30 18:59:40.697143 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:40.697113    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:40.702499 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:40.702477    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:40.716585 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:40.716561    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:40.717434 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:40.717413    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 18:59:41.021022 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:41.014578    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:42.104698 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:42.104655    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:42.792149 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:42.792113    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:43.048021 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:43.047974    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:43.236852 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:43.236819    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:46.076334 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:46.076298    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:46.281084 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:46.274155    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:47.051461 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:47.051418    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"]
Oct 30 18:59:47.055708 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:47.055690    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"
Oct 30 18:59:47.146562 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:47.146526    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrffh\" (UniqueName: \"kubernetes.io/projected/1154f650-593c-4ae1-b703-61d4b70fc24b-kube-api-access-qrffh\") pod \"maas-api-e2e-api-b-132af1-697db78c58-d8282\" (UID: \"1154f650-593c-4ae1-b703-61d4b70fc24b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"
Oct 30 18:59:47.146943 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:47.146585    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1154f650-593c-4ae1-b703-61d4b70fc24b-metrics-certs\") pod \"maas-api-e2e-api-b-132af1-697db78c58-d8282\" (UID: \"1154f650-593c-4ae1-b703-61d4b70fc24b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"
Oct 30 18:59:47.146943 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:47.146687    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1154f650-593c-4ae1-b703-61d4b70fc24b-maas-api-tls\") pod \"maas-api-e2e-api-b-132af1-697db78c58-d8282\" (UID: \"1154f650-593c-4ae1-b703-61d4b70fc24b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"
Oct 30 18:59:47.146943 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:47.146826    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1154f650-593c-4ae1-b703-61d4b70fc24b-openshift-service-ca\") pod \"maas-api-e2e-api-b-132af1-697db78c58-d8282\" (UID: \"1154f650-593c-4ae1-b703-61d4b70fc24b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"
Oct 30 18:59:47.162812 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:47.162787    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-132af1-serving-cert\""
Oct 30 18:59:47.166504 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:47.166486    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"]
Oct 30 18:59:47.248084 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:47.248049    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrffh\" (UniqueName: \"kubernetes.io/projected/1154f650-593c-4ae1-b703-61d4b70fc24b-kube-api-access-qrffh\") pod \"maas-api-e2e-api-b-132af1-697db78c58-d8282\" (UID: \"1154f650-593c-4ae1-b703-61d4b70fc24b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"
Oct 30 18:59:47.248262 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:47.248107    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1154f650-593c-4ae1-b703-61d4b70fc24b-metrics-certs\") pod \"maas-api-e2e-api-b-132af1-697db78c58-d8282\" (UID: \"1154f650-593c-4ae1-b703-61d4b70fc24b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"
Oct 30 18:59:47.248262 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:47.248179    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1154f650-593c-4ae1-b703-61d4b70fc24b-maas-api-tls\") pod \"maas-api-e2e-api-b-132af1-697db78c58-d8282\" (UID: \"1154f650-593c-4ae1-b703-61d4b70fc24b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"
Oct 30 18:59:47.248404 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:59:47.248329    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-serving-cert: secret "maas-api-e2e-api-b-132af1-serving-cert" not found
Oct 30 18:59:47.248404 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:47.248357    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1154f650-593c-4ae1-b703-61d4b70fc24b-openshift-service-ca\") pod \"maas-api-e2e-api-b-132af1-697db78c58-d8282\" (UID: \"1154f650-593c-4ae1-b703-61d4b70fc24b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"
Oct 30 18:59:47.248516 ip-10-0-133-80 kubenswrapper[5820]: E1030 18:59:47.248435    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1154f650-593c-4ae1-b703-61d4b70fc24b-maas-api-tls podName:1154f650-593c-4ae1-b703-61d4b70fc24b nodeName:}" failed. No retries permitted until 2026-10-30 18:59:47.748405076 +0000 UTC m=+1508.460153764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1154f650-593c-4ae1-b703-61d4b70fc24b-maas-api-tls") pod "maas-api-e2e-api-b-132af1-697db78c58-d8282" (UID: "1154f650-593c-4ae1-b703-61d4b70fc24b") : secret "maas-api-e2e-api-b-132af1-serving-cert" not found
Oct 30 18:59:47.249013 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:47.248991    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1154f650-593c-4ae1-b703-61d4b70fc24b-openshift-service-ca\") pod \"maas-api-e2e-api-b-132af1-697db78c58-d8282\" (UID: \"1154f650-593c-4ae1-b703-61d4b70fc24b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"
Oct 30 18:59:47.250752 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:47.250732    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1154f650-593c-4ae1-b703-61d4b70fc24b-metrics-certs\") pod \"maas-api-e2e-api-b-132af1-697db78c58-d8282\" (UID: \"1154f650-593c-4ae1-b703-61d4b70fc24b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"
Oct 30 18:59:47.330972 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:47.324686    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:47.368857 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:47.368821    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrffh\" (UniqueName: \"kubernetes.io/projected/1154f650-593c-4ae1-b703-61d4b70fc24b-kube-api-access-qrffh\") pod \"maas-api-e2e-api-b-132af1-697db78c58-d8282\" (UID: \"1154f650-593c-4ae1-b703-61d4b70fc24b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"
Oct 30 18:59:47.549693 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:47.549419    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:47.760700 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:47.760620    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1154f650-593c-4ae1-b703-61d4b70fc24b-maas-api-tls\") pod \"maas-api-e2e-api-b-132af1-697db78c58-d8282\" (UID: \"1154f650-593c-4ae1-b703-61d4b70fc24b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"
Oct 30 18:59:47.763399 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:47.763368    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1154f650-593c-4ae1-b703-61d4b70fc24b-maas-api-tls\") pod \"maas-api-e2e-api-b-132af1-697db78c58-d8282\" (UID: \"1154f650-593c-4ae1-b703-61d4b70fc24b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"
Oct 30 18:59:47.776866 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:47.776839    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:47.969866 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:47.969758    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"
Oct 30 18:59:48.495196 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:48.495166    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"]
Oct 30 18:59:48.496271 ip-10-0-133-80 kubenswrapper[5820]: W1030 18:59:48.496241    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1154f650_593c_4ae1_b703_61d4b70fc24b.slice/crio-686764cb4f19f0bf35b0e617557e0e0b1bf8559e2fc5eac804a5eb3c21b9ff91 WatchSource:0}: Error finding container 686764cb4f19f0bf35b0e617557e0e0b1bf8559e2fc5eac804a5eb3c21b9ff91: Status 404 returned error can't find the container with id 686764cb4f19f0bf35b0e617557e0e0b1bf8559e2fc5eac804a5eb3c21b9ff91
Oct 30 18:59:48.900113 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:48.900081    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282" event={"ID":"1154f650-593c-4ae1-b703-61d4b70fc24b","Type":"ContainerStarted","Data":"686764cb4f19f0bf35b0e617557e0e0b1bf8559e2fc5eac804a5eb3c21b9ff91"}
Oct 30 18:59:49.505029 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:49.504998    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:49.783361 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:49.783244    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:49.909044 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:49.909005    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282" event={"ID":"1154f650-593c-4ae1-b703-61d4b70fc24b","Type":"ContainerStarted","Data":"584c31620e5afab493102ca988c419d9db897e0841b67ddaecaae05c96fe6b3c"}
Oct 30 18:59:49.909218 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:49.909176    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"
Oct 30 18:59:50.032033 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:50.031919    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282" podStartSLOduration=2.141151189 podStartE2EDuration="3.031901176s" totalImagesPullingTime="890.749987ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:50.029772432 +0000 UTC m=+1510.741521132" watchObservedRunningTime="2026-10-30 18:59:50.031901176 +0000 UTC m=+1510.743649876"
Oct 30 18:59:50.244643 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:50.244604    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:50.492844 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:50.492721    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:53.246975 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:53.245176    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:54.858110 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:54.858069    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:55.922708 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:55.922680    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"
Oct 30 18:59:56.355071 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:56.355039    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:56.621457 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:56.620336    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:58.484979 ip-10-0-133-80 kubenswrapper[5820]: I1030 18:59:58.483790    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:00.211551 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:00.211512    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf"]
Oct 30 19:00:00.216013 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:00.215993    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf"
Oct 30 19:00:00.272038 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:00.271999    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s76d\" (UniqueName: \"kubernetes.io/projected/a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c-kube-api-access-5s76d\") pod \"maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf\" (UID: \"a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf"
Oct 30 19:00:00.272256 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:00.272233    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf\" (UID: \"a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf"
Oct 30 19:00:00.333846 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:00.333812    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qg7n\""
Oct 30 19:00:00.338064 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:00.338030    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf"]
Oct 30 19:00:00.373385 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:00.373352    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s76d\" (UniqueName: \"kubernetes.io/projected/a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c-kube-api-access-5s76d\") pod \"maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf\" (UID: \"a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf"
Oct 30 19:00:00.373566 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:00.373509    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf\" (UID: \"a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf"
Oct 30 19:00:00.374286 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:00.374258    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf\" (UID: \"a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf"
Oct 30 19:00:00.435225 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:00.435192    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s76d\" (UniqueName: \"kubernetes.io/projected/a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c-kube-api-access-5s76d\") pod \"maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf\" (UID: \"a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf"
Oct 30 19:00:00.531220 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:00.531144    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf"
Oct 30 19:00:00.944436 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:00.944405    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf"]
Oct 30 19:00:00.946480 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:00:00.946425    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda36e9cac_95e2_4e7e_8c7a_fab0e6c39d5c.slice/crio-28c45efcb438f195a5f236f192ae2b3d99ab1713193f59e27ed2a3e704441283 WatchSource:0}: Error finding container 28c45efcb438f195a5f236f192ae2b3d99ab1713193f59e27ed2a3e704441283: Status 404 returned error can't find the container with id 28c45efcb438f195a5f236f192ae2b3d99ab1713193f59e27ed2a3e704441283
Oct 30 19:00:01.128864 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:01.128824    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf" event={"ID":"a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c","Type":"ContainerStarted","Data":"28c45efcb438f195a5f236f192ae2b3d99ab1713193f59e27ed2a3e704441283"}
Oct 30 19:00:03.582419 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:03.574146    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:06.178365 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:06.178336    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf" event={"ID":"a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c","Type":"ContainerStarted","Data":"a6c1e47b2b8c2c77fe8165d3baf8b98251708cbc48fd1696224dbd16856f096e"}
Oct 30 19:00:06.294677 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:06.294622    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf" podStartSLOduration=1.37297087 podStartE2EDuration="6.294607636s" totalImagesPullingTime="4.921636766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:06.292584973 +0000 UTC m=+1527.004333670" watchObservedRunningTime="2026-10-30 19:00:06.294607636 +0000 UTC m=+1527.006356332"
Oct 30 19:00:07.186148 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:07.186113    5820 generic.go:356] "Generic (PLEG): container finished" podID="a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c" containerID="a6c1e47b2b8c2c77fe8165d3baf8b98251708cbc48fd1696224dbd16856f096e" exitCode=60
Oct 30 19:00:07.186570 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:07.186202    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf" event={"ID":"a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c","Type":"ContainerDied","Data":"a6c1e47b2b8c2c77fe8165d3baf8b98251708cbc48fd1696224dbd16856f096e"}
Oct 30 19:00:07.186570 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:07.186430    5820 scope.go:122] "RemoveContainer" containerID="a6c1e47b2b8c2c77fe8165d3baf8b98251708cbc48fd1696224dbd16856f096e"
Oct 30 19:00:08.196640 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:08.196597    5820 generic.go:356] "Generic (PLEG): container finished" podID="a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c" containerID="4ab9858647127e7456717a72b8610771f3cc842feb25fa37627f47717191bee2" exitCode=60
Oct 30 19:00:08.197112 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:08.196650    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf" event={"ID":"a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c","Type":"ContainerDied","Data":"4ab9858647127e7456717a72b8610771f3cc842feb25fa37627f47717191bee2"}
Oct 30 19:00:08.197112 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:08.196721    5820 scope.go:122] "RemoveContainer" containerID="a6c1e47b2b8c2c77fe8165d3baf8b98251708cbc48fd1696224dbd16856f096e"
Oct 30 19:00:08.197112 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:08.197067    5820 scope.go:122] "RemoveContainer" containerID="4ab9858647127e7456717a72b8610771f3cc842feb25fa37627f47717191bee2"
Oct 30 19:00:08.197409 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:00:08.197377    5820 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf_odh-ai-gateway-infra(a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf" podUID="a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c"
Oct 30 19:00:08.205888 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:00:08.205857    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:00:08.486975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:08.483674    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:09.205888 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:09.205860    5820 scope.go:122] "RemoveContainer" containerID="4ab9858647127e7456717a72b8610771f3cc842feb25fa37627f47717191bee2"
Oct 30 19:00:09.206468 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:00:09.206396    5820 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf_odh-ai-gateway-infra(a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf" podUID="a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c"
Oct 30 19:00:10.938641 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:10.938604    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"]
Oct 30 19:00:10.939136 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:10.938788    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48" podUID="180fc1d4-d421-446a-afd7-f1f59bfafe0f" containerName="istio-proxy" containerID="cri-o://9ee0872e94052e046652ebecab0c991b075fc7ab7a2e18814f9eea958d629189" gracePeriod=30
Oct 30 19:00:10.988448 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:10.988413    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"]
Oct 30 19:00:10.988634 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:10.988612    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd" podUID="1057f570-7bb3-4bfb-ab81-6696e3b792b2" containerName="istio-proxy" containerID="cri-o://8df6b203ba45fcdbd4c34038515e9ce131e5a947a041b02ca5f093b78d708708" gracePeriod=30
Oct 30 19:00:13.642013 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:13.623848    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:15.896768 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:15.896733    5820 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:00:15.897186 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:15.896796    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48" podUID="180fc1d4-d421-446a-afd7-f1f59bfafe0f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:00:16.255155 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.255119    5820 generic.go:356] "Generic (PLEG): container finished" podID="180fc1d4-d421-446a-afd7-f1f59bfafe0f" containerID="9ee0872e94052e046652ebecab0c991b075fc7ab7a2e18814f9eea958d629189" exitCode=0
Oct 30 19:00:16.255344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.255152    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48" event={"ID":"180fc1d4-d421-446a-afd7-f1f59bfafe0f","Type":"ContainerDied","Data":"9ee0872e94052e046652ebecab0c991b075fc7ab7a2e18814f9eea958d629189"}
Oct 30 19:00:16.255344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.255220    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48" event={"ID":"180fc1d4-d421-446a-afd7-f1f59bfafe0f","Type":"ContainerDied","Data":"d9f76d8de9b54af4f988d82336a46b4878e8badf5568f01505af256436760d72"}
Oct 30 19:00:16.255344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.255247    5820 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d9f76d8de9b54af4f988d82336a46b4878e8badf5568f01505af256436760d72"
Oct 30 19:00:16.257450 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.257422    5820 generic.go:356] "Generic (PLEG): container finished" podID="1057f570-7bb3-4bfb-ab81-6696e3b792b2" containerID="8df6b203ba45fcdbd4c34038515e9ce131e5a947a041b02ca5f093b78d708708" exitCode=0
Oct 30 19:00:16.257635 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.257611    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd" event={"ID":"1057f570-7bb3-4bfb-ab81-6696e3b792b2","Type":"ContainerDied","Data":"8df6b203ba45fcdbd4c34038515e9ce131e5a947a041b02ca5f093b78d708708"}
Oct 30 19:00:16.273411 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.273385    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 19:00:16.328430 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.328407    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 19:00:16.412564 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.412484    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-token\" (UniqueName: \"kubernetes.io/projected/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-token\") pod \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") "
Oct 30 19:00:16.412725 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.412577    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istiod-ca-cert\") pod \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") "
Oct 30 19:00:16.412725 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.412618    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-workload-certs\") pod \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") "
Oct 30 19:00:16.412725 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.412654    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1057f570-7bb3-4bfb-ab81-6696e3b792b2-kube-api-access-pvqqj\" (UniqueName: \"kubernetes.io/projected/1057f570-7bb3-4bfb-ab81-6696e3b792b2-kube-api-access-pvqqj\") pod \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") "
Oct 30 19:00:16.412725 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.412698    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-envoy\") pod \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") "
Oct 30 19:00:16.412943 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.412744    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-data\") pod \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") "
Oct 30 19:00:16.412943 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.412801    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-workload-socket\") pod \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") "
Oct 30 19:00:16.412943 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.412840    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-podinfo\") pod \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") "
Oct 30 19:00:16.412943 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.412896    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-workload-socket\") pod \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") "
Oct 30 19:00:16.413212 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.412972    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-workload-certs" pod "1057f570-7bb3-4bfb-ab81-6696e3b792b2" (UID: "1057f570-7bb3-4bfb-ab81-6696e3b792b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:16.413212 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.412940    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/180fc1d4-d421-446a-afd7-f1f59bfafe0f-kube-api-access-zgtw8\" (UniqueName: \"kubernetes.io/projected/180fc1d4-d421-446a-afd7-f1f59bfafe0f-kube-api-access-zgtw8\") pod \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") "
Oct 30 19:00:16.413212 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.413045    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istiod-ca-cert" pod "1057f570-7bb3-4bfb-ab81-6696e3b792b2" (UID: "1057f570-7bb3-4bfb-ab81-6696e3b792b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:16.413212 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.413101    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-data" pod "1057f570-7bb3-4bfb-ab81-6696e3b792b2" (UID: "1057f570-7bb3-4bfb-ab81-6696e3b792b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:16.413212 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.413151    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-credential-socket\") pod \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") "
Oct 30 19:00:16.413569 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.413219    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-workload-certs\") pod \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") "
Oct 30 19:00:16.413569 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.413275    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-podinfo\") pod \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") "
Oct 30 19:00:16.413569 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.413301    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-workload-socket" pod "1057f570-7bb3-4bfb-ab81-6696e3b792b2" (UID: "1057f570-7bb3-4bfb-ab81-6696e3b792b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:16.413569 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.413331    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-envoy\") pod \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") "
Oct 30 19:00:16.413569 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.413497    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-workload-socket" pod "180fc1d4-d421-446a-afd7-f1f59bfafe0f" (UID: "180fc1d4-d421-446a-afd7-f1f59bfafe0f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:16.413569 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.413603    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-credential-socket\") pod \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\" (UID: \"1057f570-7bb3-4bfb-ab81-6696e3b792b2\") "
Oct 30 19:00:16.414101 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.413660    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-token\" (UniqueName: \"kubernetes.io/projected/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-token\") pod \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") "
Oct 30 19:00:16.414101 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.413673    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-workload-certs" pod "180fc1d4-d421-446a-afd7-f1f59bfafe0f" (UID: "180fc1d4-d421-446a-afd7-f1f59bfafe0f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:16.414101 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.413719    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-data\") pod \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") "
Oct 30 19:00:16.414101 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.413754    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istiod-ca-cert\") pod \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\" (UID: \"180fc1d4-d421-446a-afd7-f1f59bfafe0f\") "
Oct 30 19:00:16.414101 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.414040    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-credential-socket" pod "180fc1d4-d421-446a-afd7-f1f59bfafe0f" (UID: "180fc1d4-d421-446a-afd7-f1f59bfafe0f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:16.414377 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.414223    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-credential-socket" pod "1057f570-7bb3-4bfb-ab81-6696e3b792b2" (UID: "1057f570-7bb3-4bfb-ab81-6696e3b792b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:16.414551 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.414527    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istiod-ca-cert" pod "180fc1d4-d421-446a-afd7-f1f59bfafe0f" (UID: "180fc1d4-d421-446a-afd7-f1f59bfafe0f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:16.414652 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.414628    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:16.414713 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.414660    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:16.414713 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.414677    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:16.414713 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.414691    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:16.414713 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.414706    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:16.414930 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.414720    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:16.414930 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.414734    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:16.414930 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.414747    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:16.415797 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.415683    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1057f570-7bb3-4bfb-ab81-6696e3b792b2-kube-api-access-pvqqj" pod "1057f570-7bb3-4bfb-ab81-6696e3b792b2" (UID: "1057f570-7bb3-4bfb-ab81-6696e3b792b2"). InnerVolumeSpecName "kube-api-access-pvqqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:16.415797 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.415729    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-token" pod "1057f570-7bb3-4bfb-ab81-6696e3b792b2" (UID: "1057f570-7bb3-4bfb-ab81-6696e3b792b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:16.415797 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.415742    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-envoy" pod "180fc1d4-d421-446a-afd7-f1f59bfafe0f" (UID: "180fc1d4-d421-446a-afd7-f1f59bfafe0f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:16.416058 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.415870    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/180fc1d4-d421-446a-afd7-f1f59bfafe0f-kube-api-access-zgtw8" pod "180fc1d4-d421-446a-afd7-f1f59bfafe0f" (UID: "180fc1d4-d421-446a-afd7-f1f59bfafe0f"). InnerVolumeSpecName "kube-api-access-zgtw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:16.416058 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.415925    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-data" pod "180fc1d4-d421-446a-afd7-f1f59bfafe0f" (UID: "180fc1d4-d421-446a-afd7-f1f59bfafe0f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:16.416058 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.416024    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-podinfo" pod "1057f570-7bb3-4bfb-ab81-6696e3b792b2" (UID: "1057f570-7bb3-4bfb-ab81-6696e3b792b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:00:16.416634 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.416615    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-token" pod "180fc1d4-d421-446a-afd7-f1f59bfafe0f" (UID: "180fc1d4-d421-446a-afd7-f1f59bfafe0f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:16.417015 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.416994    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-podinfo" pod "180fc1d4-d421-446a-afd7-f1f59bfafe0f" (UID: "180fc1d4-d421-446a-afd7-f1f59bfafe0f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:00:16.417186 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.417168    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-envoy" pod "1057f570-7bb3-4bfb-ab81-6696e3b792b2" (UID: "1057f570-7bb3-4bfb-ab81-6696e3b792b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:16.516052 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.516013    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvqqj\" (UniqueName: \"kubernetes.io/projected/1057f570-7bb3-4bfb-ab81-6696e3b792b2-kube-api-access-pvqqj\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:16.516052 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.516050    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:16.516251 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.516064    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:16.516251 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.516077    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgtw8\" (UniqueName: \"kubernetes.io/projected/180fc1d4-d421-446a-afd7-f1f59bfafe0f-kube-api-access-zgtw8\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:16.516251 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.516093    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:16.516251 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.516105    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:16.516251 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.516118    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:16.516251 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.516132    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:16.516251 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.516145    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/180fc1d4-d421-446a-afd7-f1f59bfafe0f-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:16.516251 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.516160    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1057f570-7bb3-4bfb-ab81-6696e3b792b2-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:16.649978 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.649921    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"]
Oct 30 19:00:16.650251 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.650218    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc" podUID="0378c741-36a6-4c68-97bb-f3c6f4ec3470" containerName="istio-proxy" containerID="cri-o://4e0cd1fe656371a287b55fd9340a0b73a1b8faa9570caf654416b39c35bac92c" gracePeriod=30
Oct 30 19:00:16.678110 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.678028    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"]
Oct 30 19:00:16.678291 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.678258    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw" podUID="036d7b93-15ba-499b-9ed5-a4217c856742" containerName="istio-proxy" containerID="cri-o://56de44c8af394d2e1b66452336dfa3f45613827ca0ff184be4f5a080bf487366" gracePeriod=30
Oct 30 19:00:16.681347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.681305    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"]
Oct 30 19:00:16.681559 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.681518    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8" podUID="142d862b-ea80-4125-b065-7c3d471fe400" containerName="istio-proxy" containerID="cri-o://492e2c395aae3fa51c232eaa9bc21adb8734e3a721846da663dcb9cf8976f621" gracePeriod=30
Oct 30 19:00:16.684099 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.684076    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"]
Oct 30 19:00:16.684320 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.684274    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs" podUID="72548522-bba4-4f29-92ea-4d41d72e88b1" containerName="istio-proxy" containerID="cri-o://89dd0215184eaf3a324caddfa14a4745bfaa69895726f93d93c6d823e6269eea" gracePeriod=30
Oct 30 19:00:16.696160 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.696114    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"]
Oct 30 19:00:16.696351 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:16.696327    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796" podUID="29ce830e-8096-4239-8ad4-03163e8fc5dc" containerName="istio-proxy" containerID="cri-o://def293b92810a87ddde97b30ad90ddcff6ebda969bf0871b9dda302d9c74cf69" gracePeriod=30
Oct 30 19:00:17.265537 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:17.265439    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"
Oct 30 19:00:17.265537 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:17.265478    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd" event={"ID":"1057f570-7bb3-4bfb-ab81-6696e3b792b2","Type":"ContainerDied","Data":"a2b2d0bdebf6e5303bc6ad51c0882dd525c35638920fdc7a10ee1f3d0a85a402"}
Oct 30 19:00:17.266076 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:17.265541    5820 scope.go:122] "RemoveContainer" containerID="8df6b203ba45fcdbd4c34038515e9ce131e5a947a041b02ca5f093b78d708708"
Oct 30 19:00:17.266076 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:17.265689    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"
Oct 30 19:00:17.428414 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:17.428380    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"]
Oct 30 19:00:17.438939 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:17.438912    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-x5dwd"]
Oct 30 19:00:17.551588 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:17.551558    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"]
Oct 30 19:00:17.554888 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:17.554867    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-zwd48"]
Oct 30 19:00:18.429438 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:18.429404    5820 scope.go:122] "RemoveContainer" containerID="4ab9858647127e7456717a72b8610771f3cc842feb25fa37627f47717191bee2"
Oct 30 19:00:18.479855 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:18.479822    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:19.281883 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:19.281845    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf" event={"ID":"a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c","Type":"ContainerStarted","Data":"bde1cc902caac9e5de579bfa6fc2873b70ef697f39faa14202e89f189d9fc08b"}
Oct 30 19:00:19.438668 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:19.438632    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1057f570-7bb3-4bfb-ab81-6696e3b792b2" path="/var/lib/kubelet/pods/1057f570-7bb3-4bfb-ab81-6696e3b792b2/volumes"
Oct 30 19:00:19.439211 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:19.439195    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="180fc1d4-d421-446a-afd7-f1f59bfafe0f" path="/var/lib/kubelet/pods/180fc1d4-d421-446a-afd7-f1f59bfafe0f/volumes"
Oct 30 19:00:20.291874 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:20.291844    5820 generic.go:356] "Generic (PLEG): container finished" podID="a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c" containerID="bde1cc902caac9e5de579bfa6fc2873b70ef697f39faa14202e89f189d9fc08b" exitCode=60
Oct 30 19:00:20.292082 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:20.291931    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf" event={"ID":"a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c","Type":"ContainerDied","Data":"bde1cc902caac9e5de579bfa6fc2873b70ef697f39faa14202e89f189d9fc08b"}
Oct 30 19:00:20.292082 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:20.292014    5820 scope.go:122] "RemoveContainer" containerID="4ab9858647127e7456717a72b8610771f3cc842feb25fa37627f47717191bee2"
Oct 30 19:00:20.292422 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:20.292400    5820 scope.go:122] "RemoveContainer" containerID="bde1cc902caac9e5de579bfa6fc2873b70ef697f39faa14202e89f189d9fc08b"
Oct 30 19:00:20.292744 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:00:20.292724    5820 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf_odh-ai-gateway-infra(a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf" podUID="a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c"
Oct 30 19:00:20.437889 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:20.437856    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf"]
Oct 30 19:00:20.752305 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:20.752266    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:00:20.752825 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:20.752328    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs" podUID="72548522-bba4-4f29-92ea-4d41d72e88b1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:00:21.487093 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:21.487070    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf"
Oct 30 19:00:21.596983 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:21.596884    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c-kube-api-access-5s76d\" (UniqueName: \"kubernetes.io/projected/a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c-kube-api-access-5s76d\") pod \"a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c\" (UID: \"a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c\") "
Oct 30 19:00:21.597143 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:21.596990    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c-maas-api-service-ca\") pod \"a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c\" (UID: \"a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c\") "
Oct 30 19:00:21.597342 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:21.597318    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c-maas-api-service-ca" pod "a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c" (UID: "a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:21.597473 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:21.597460    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c-maas-api-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:21.599275 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:21.599246    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c-kube-api-access-5s76d" pod "a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c" (UID: "a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c"). InnerVolumeSpecName "kube-api-access-5s76d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:21.611379 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:21.611355    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:00:21.611477 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:21.611414    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796" podUID="29ce830e-8096-4239-8ad4-03163e8fc5dc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:00:21.699510 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:21.699476    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5s76d\" (UniqueName: \"kubernetes.io/projected/a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c-kube-api-access-5s76d\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.029893 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.029574    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 19:00:22.101546 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.101401    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 19:00:22.103640 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.103616    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 19:00:22.110991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.110930    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 19:00:22.171541 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.171514    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 19:00:22.208652 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.208623    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0378c741-36a6-4c68-97bb-f3c6f4ec3470-kube-api-access-kn7vn\" (UniqueName: \"kubernetes.io/projected/0378c741-36a6-4c68-97bb-f3c6f4ec3470-kube-api-access-kn7vn\") pod \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") "
Oct 30 19:00:22.208832 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.208666    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/036d7b93-15ba-499b-9ed5-a4217c856742-istio-token\" (UniqueName: \"kubernetes.io/projected/036d7b93-15ba-499b-9ed5-a4217c856742-istio-token\") pod \"036d7b93-15ba-499b-9ed5-a4217c856742\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") "
Oct 30 19:00:22.208832 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.208698    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-credential-socket\") pod \"036d7b93-15ba-499b-9ed5-a4217c856742\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") "
Oct 30 19:00:22.208832 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.208724    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/142d862b-ea80-4125-b065-7c3d471fe400-kube-api-access-r9lrq\" (UniqueName: \"kubernetes.io/projected/142d862b-ea80-4125-b065-7c3d471fe400-kube-api-access-r9lrq\") pod \"142d862b-ea80-4125-b065-7c3d471fe400\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") "
Oct 30 19:00:22.208832 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.208752    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-credential-socket\") pod \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") "
Oct 30 19:00:22.208832 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.208780    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-workload-certs\") pod \"72548522-bba4-4f29-92ea-4d41d72e88b1\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") "
Oct 30 19:00:22.208832 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.208813    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-data\") pod \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") "
Oct 30 19:00:22.209161 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.208847    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/142d862b-ea80-4125-b065-7c3d471fe400-istio-token\" (UniqueName: \"kubernetes.io/projected/142d862b-ea80-4125-b065-7c3d471fe400-istio-token\") pod \"142d862b-ea80-4125-b065-7c3d471fe400\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") "
Oct 30 19:00:22.209161 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.208897    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-workload-socket\") pod \"72548522-bba4-4f29-92ea-4d41d72e88b1\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") "
Oct 30 19:00:22.209161 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.208929    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-workload-socket\") pod \"142d862b-ea80-4125-b065-7c3d471fe400\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") "
Oct 30 19:00:22.209161 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.209028    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/036d7b93-15ba-499b-9ed5-a4217c856742-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/036d7b93-15ba-499b-9ed5-a4217c856742-istio-podinfo\") pod \"036d7b93-15ba-499b-9ed5-a4217c856742\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") "
Oct 30 19:00:22.209161 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.209075    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-credential-socket" pod "0378c741-36a6-4c68-97bb-f3c6f4ec3470" (UID: "0378c741-36a6-4c68-97bb-f3c6f4ec3470"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.209161 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.209156    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-istio-envoy\") pod \"142d862b-ea80-4125-b065-7c3d471fe400\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") "
Oct 30 19:00:22.209512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.209275    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-data" pod "0378c741-36a6-4c68-97bb-f3c6f4ec3470" (UID: "0378c741-36a6-4c68-97bb-f3c6f4ec3470"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.209512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.209291    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-podinfo\") pod \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") "
Oct 30 19:00:22.209512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.209350    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/72548522-bba4-4f29-92ea-4d41d72e88b1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72548522-bba4-4f29-92ea-4d41d72e88b1-istiod-ca-cert\") pod \"72548522-bba4-4f29-92ea-4d41d72e88b1\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") "
Oct 30 19:00:22.209512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.209395    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-workload-certs\") pod \"036d7b93-15ba-499b-9ed5-a4217c856742\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") "
Oct 30 19:00:22.209512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.209411    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-workload-certs" pod "72548522-bba4-4f29-92ea-4d41d72e88b1" (UID: "72548522-bba4-4f29-92ea-4d41d72e88b1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.209512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.209477    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-token\" (UniqueName: \"kubernetes.io/projected/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-token\") pod \"72548522-bba4-4f29-92ea-4d41d72e88b1\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") "
Oct 30 19:00:22.209512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.209530    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-podinfo\") pod \"72548522-bba4-4f29-92ea-4d41d72e88b1\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") "
Oct 30 19:00:22.209938 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.209574    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-credential-socket\") pod \"72548522-bba4-4f29-92ea-4d41d72e88b1\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") "
Oct 30 19:00:22.209938 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.209603    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-istio-data\") pod \"142d862b-ea80-4125-b065-7c3d471fe400\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") "
Oct 30 19:00:22.209938 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.209645    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-workload-socket" pod "142d862b-ea80-4125-b065-7c3d471fe400" (UID: "142d862b-ea80-4125-b065-7c3d471fe400"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.209938 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.209652    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/142d862b-ea80-4125-b065-7c3d471fe400-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/142d862b-ea80-4125-b065-7c3d471fe400-istiod-ca-cert\") pod \"142d862b-ea80-4125-b065-7c3d471fe400\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") "
Oct 30 19:00:22.209938 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.209674    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-workload-socket" pod "72548522-bba4-4f29-92ea-4d41d72e88b1" (UID: "72548522-bba4-4f29-92ea-4d41d72e88b1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.209938 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.209721    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-credential-socket" pod "036d7b93-15ba-499b-9ed5-a4217c856742" (UID: "036d7b93-15ba-499b-9ed5-a4217c856742"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.209938 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.209756    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-envoy\") pod \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") "
Oct 30 19:00:22.209938 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.209901    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-envoy\") pod \"72548522-bba4-4f29-92ea-4d41d72e88b1\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") "
Oct 30 19:00:22.210355 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.209975    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/72548522-bba4-4f29-92ea-4d41d72e88b1-kube-api-access-j7drp\" (UniqueName: \"kubernetes.io/projected/72548522-bba4-4f29-92ea-4d41d72e88b1-kube-api-access-j7drp\") pod \"72548522-bba4-4f29-92ea-4d41d72e88b1\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") "
Oct 30 19:00:22.210355 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.210008    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-istio-data\") pod \"036d7b93-15ba-499b-9ed5-a4217c856742\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") "
Oct 30 19:00:22.210355 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.210022    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/142d862b-ea80-4125-b065-7c3d471fe400-istiod-ca-cert" pod "142d862b-ea80-4125-b065-7c3d471fe400" (UID: "142d862b-ea80-4125-b065-7c3d471fe400"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:22.210355 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.210085    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-workload-certs\") pod \"142d862b-ea80-4125-b065-7c3d471fe400\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") "
Oct 30 19:00:22.210355 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.210129    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-istio-envoy\") pod \"036d7b93-15ba-499b-9ed5-a4217c856742\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") "
Oct 30 19:00:22.210355 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.210265    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/142d862b-ea80-4125-b065-7c3d471fe400-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/142d862b-ea80-4125-b065-7c3d471fe400-istio-podinfo\") pod \"142d862b-ea80-4125-b065-7c3d471fe400\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") "
Oct 30 19:00:22.210355 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.210274    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-istio-data" pod "036d7b93-15ba-499b-9ed5-a4217c856742" (UID: "036d7b93-15ba-499b-9ed5-a4217c856742"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.210355 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.210293    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-workload-certs" pod "036d7b93-15ba-499b-9ed5-a4217c856742" (UID: "036d7b93-15ba-499b-9ed5-a4217c856742"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.210355 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.210299    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-workload-socket\") pod \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") "
Oct 30 19:00:22.210837 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.210481    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-workload-socket" pod "0378c741-36a6-4c68-97bb-f3c6f4ec3470" (UID: "0378c741-36a6-4c68-97bb-f3c6f4ec3470"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.210837 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.210634    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istiod-ca-cert\") pod \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") "
Oct 30 19:00:22.210837 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.210695    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-data\") pod \"72548522-bba4-4f29-92ea-4d41d72e88b1\" (UID: \"72548522-bba4-4f29-92ea-4d41d72e88b1\") "
Oct 30 19:00:22.210837 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.210723    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-workload-certs\") pod \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") "
Oct 30 19:00:22.210837 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.210789    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-credential-socket\") pod \"142d862b-ea80-4125-b065-7c3d471fe400\" (UID: \"142d862b-ea80-4125-b065-7c3d471fe400\") "
Oct 30 19:00:22.210837 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.210826    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-token\" (UniqueName: \"kubernetes.io/projected/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-token\") pod \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\" (UID: \"0378c741-36a6-4c68-97bb-f3c6f4ec3470\") "
Oct 30 19:00:22.211324 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.210873    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/036d7b93-15ba-499b-9ed5-a4217c856742-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/036d7b93-15ba-499b-9ed5-a4217c856742-istiod-ca-cert\") pod \"036d7b93-15ba-499b-9ed5-a4217c856742\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") "
Oct 30 19:00:22.211324 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.210923    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/036d7b93-15ba-499b-9ed5-a4217c856742-kube-api-access-8kc9t\" (UniqueName: \"kubernetes.io/projected/036d7b93-15ba-499b-9ed5-a4217c856742-kube-api-access-8kc9t\") pod \"036d7b93-15ba-499b-9ed5-a4217c856742\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") "
Oct 30 19:00:22.211324 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.210975    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-workload-socket\") pod \"036d7b93-15ba-499b-9ed5-a4217c856742\" (UID: \"036d7b93-15ba-499b-9ed5-a4217c856742\") "
Oct 30 19:00:22.212099 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.212070    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.212192 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.212100    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.212192 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.212116    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.212192 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.212131    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.212192 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.212145    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.212192 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.212162    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.212192 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.212177    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.212393 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.212191    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/142d862b-ea80-4125-b065-7c3d471fe400-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.212393 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.212208    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.212393 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.212221    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.212523 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.212463    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-workload-socket" pod "036d7b93-15ba-499b-9ed5-a4217c856742" (UID: "036d7b93-15ba-499b-9ed5-a4217c856742"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.212898 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.212735    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-workload-certs" pod "0378c741-36a6-4c68-97bb-f3c6f4ec3470" (UID: "0378c741-36a6-4c68-97bb-f3c6f4ec3470"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.213008 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.212964    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-credential-socket" pod "72548522-bba4-4f29-92ea-4d41d72e88b1" (UID: "72548522-bba4-4f29-92ea-4d41d72e88b1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.213491 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.213155    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istiod-ca-cert" pod "0378c741-36a6-4c68-97bb-f3c6f4ec3470" (UID: "0378c741-36a6-4c68-97bb-f3c6f4ec3470"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:22.213491 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.213161    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-istio-data" pod "142d862b-ea80-4125-b065-7c3d471fe400" (UID: "142d862b-ea80-4125-b065-7c3d471fe400"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.213491 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.213325    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-data" pod "72548522-bba4-4f29-92ea-4d41d72e88b1" (UID: "72548522-bba4-4f29-92ea-4d41d72e88b1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.213491 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.213332    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72548522-bba4-4f29-92ea-4d41d72e88b1-istiod-ca-cert" pod "72548522-bba4-4f29-92ea-4d41d72e88b1" (UID: "72548522-bba4-4f29-92ea-4d41d72e88b1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:22.213491 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.213351    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-envoy" pod "72548522-bba4-4f29-92ea-4d41d72e88b1" (UID: "72548522-bba4-4f29-92ea-4d41d72e88b1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.213491 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.213374    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-workload-certs" pod "142d862b-ea80-4125-b065-7c3d471fe400" (UID: "142d862b-ea80-4125-b065-7c3d471fe400"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.214055 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.213560    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-credential-socket" pod "142d862b-ea80-4125-b065-7c3d471fe400" (UID: "142d862b-ea80-4125-b065-7c3d471fe400"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.214055 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.213632    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/036d7b93-15ba-499b-9ed5-a4217c856742-istiod-ca-cert" pod "036d7b93-15ba-499b-9ed5-a4217c856742" (UID: "036d7b93-15ba-499b-9ed5-a4217c856742"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:22.214222 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.214202    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/036d7b93-15ba-499b-9ed5-a4217c856742-istio-podinfo" pod "036d7b93-15ba-499b-9ed5-a4217c856742" (UID: "036d7b93-15ba-499b-9ed5-a4217c856742"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:00:22.214350 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.214328    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/036d7b93-15ba-499b-9ed5-a4217c856742-istio-token" pod "036d7b93-15ba-499b-9ed5-a4217c856742" (UID: "036d7b93-15ba-499b-9ed5-a4217c856742"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:22.214522 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.214498    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0378c741-36a6-4c68-97bb-f3c6f4ec3470-kube-api-access-kn7vn" pod "0378c741-36a6-4c68-97bb-f3c6f4ec3470" (UID: "0378c741-36a6-4c68-97bb-f3c6f4ec3470"). InnerVolumeSpecName "kube-api-access-kn7vn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:22.215067 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.215034    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-envoy" pod "0378c741-36a6-4c68-97bb-f3c6f4ec3470" (UID: "0378c741-36a6-4c68-97bb-f3c6f4ec3470"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.215067 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.215055    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/142d862b-ea80-4125-b065-7c3d471fe400-istio-token" pod "142d862b-ea80-4125-b065-7c3d471fe400" (UID: "142d862b-ea80-4125-b065-7c3d471fe400"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:22.216210 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.216181    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72548522-bba4-4f29-92ea-4d41d72e88b1-kube-api-access-j7drp" pod "72548522-bba4-4f29-92ea-4d41d72e88b1" (UID: "72548522-bba4-4f29-92ea-4d41d72e88b1"). InnerVolumeSpecName "kube-api-access-j7drp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:22.216317 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.216213    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/142d862b-ea80-4125-b065-7c3d471fe400-istio-podinfo" pod "142d862b-ea80-4125-b065-7c3d471fe400" (UID: "142d862b-ea80-4125-b065-7c3d471fe400"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:00:22.216317 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.216218    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-token" pod "0378c741-36a6-4c68-97bb-f3c6f4ec3470" (UID: "0378c741-36a6-4c68-97bb-f3c6f4ec3470"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:22.216317 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.216231    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-istio-envoy" pod "142d862b-ea80-4125-b065-7c3d471fe400" (UID: "142d862b-ea80-4125-b065-7c3d471fe400"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.216317 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.216267    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/142d862b-ea80-4125-b065-7c3d471fe400-kube-api-access-r9lrq" pod "142d862b-ea80-4125-b065-7c3d471fe400" (UID: "142d862b-ea80-4125-b065-7c3d471fe400"). InnerVolumeSpecName "kube-api-access-r9lrq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:22.216317 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.216281    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-podinfo" pod "72548522-bba4-4f29-92ea-4d41d72e88b1" (UID: "72548522-bba4-4f29-92ea-4d41d72e88b1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:00:22.216317 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.216279    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-token" pod "72548522-bba4-4f29-92ea-4d41d72e88b1" (UID: "72548522-bba4-4f29-92ea-4d41d72e88b1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:22.216548 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.216342    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-istio-envoy" pod "036d7b93-15ba-499b-9ed5-a4217c856742" (UID: "036d7b93-15ba-499b-9ed5-a4217c856742"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.216666 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.216650    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-podinfo" pod "0378c741-36a6-4c68-97bb-f3c6f4ec3470" (UID: "0378c741-36a6-4c68-97bb-f3c6f4ec3470"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:00:22.217907 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.217882    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/036d7b93-15ba-499b-9ed5-a4217c856742-kube-api-access-8kc9t" pod "036d7b93-15ba-499b-9ed5-a4217c856742" (UID: "036d7b93-15ba-499b-9ed5-a4217c856742"). InnerVolumeSpecName "kube-api-access-8kc9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:22.312281 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.312143    5820 generic.go:356] "Generic (PLEG): container finished" podID="036d7b93-15ba-499b-9ed5-a4217c856742" containerID="56de44c8af394d2e1b66452336dfa3f45613827ca0ff184be4f5a080bf487366" exitCode=0
Oct 30 19:00:22.312483 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.312188    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw" event={"ID":"036d7b93-15ba-499b-9ed5-a4217c856742","Type":"ContainerDied","Data":"56de44c8af394d2e1b66452336dfa3f45613827ca0ff184be4f5a080bf487366"}
Oct 30 19:00:22.312832 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.312644    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw" event={"ID":"036d7b93-15ba-499b-9ed5-a4217c856742","Type":"ContainerDied","Data":"3e8f3d666d992cb31cbdafc3bb32b873f8d64da7bb81f98cf8ed75d26fd01013"}
Oct 30 19:00:22.312832 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.312701    5820 scope.go:122] "RemoveContainer" containerID="56de44c8af394d2e1b66452336dfa3f45613827ca0ff184be4f5a080bf487366"
Oct 30 19:00:22.312832 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.312201    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"
Oct 30 19:00:22.314195 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.314174    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-credential-socket\") pod \"29ce830e-8096-4239-8ad4-03163e8fc5dc\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") "
Oct 30 19:00:22.314325 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.314241    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-podinfo\") pod \"29ce830e-8096-4239-8ad4-03163e8fc5dc\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") "
Oct 30 19:00:22.314750 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.314329    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29ce830e-8096-4239-8ad4-03163e8fc5dc-kube-api-access-hjgwk\" (UniqueName: \"kubernetes.io/projected/29ce830e-8096-4239-8ad4-03163e8fc5dc-kube-api-access-hjgwk\") pod \"29ce830e-8096-4239-8ad4-03163e8fc5dc\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") "
Oct 30 19:00:22.314750 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.314349    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-envoy\") pod \"29ce830e-8096-4239-8ad4-03163e8fc5dc\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") "
Oct 30 19:00:22.314750 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.314379    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-token\" (UniqueName: \"kubernetes.io/projected/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-token\") pod \"29ce830e-8096-4239-8ad4-03163e8fc5dc\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") "
Oct 30 19:00:22.314750 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.314400    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-data\") pod \"29ce830e-8096-4239-8ad4-03163e8fc5dc\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") "
Oct 30 19:00:22.314750 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.314445    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29ce830e-8096-4239-8ad4-03163e8fc5dc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29ce830e-8096-4239-8ad4-03163e8fc5dc-istiod-ca-cert\") pod \"29ce830e-8096-4239-8ad4-03163e8fc5dc\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") "
Oct 30 19:00:22.314750 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.314527    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-workload-certs\") pod \"29ce830e-8096-4239-8ad4-03163e8fc5dc\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") "
Oct 30 19:00:22.314750 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.314562    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-workload-socket\") pod \"29ce830e-8096-4239-8ad4-03163e8fc5dc\" (UID: \"29ce830e-8096-4239-8ad4-03163e8fc5dc\") "
Oct 30 19:00:22.315187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.314984    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.315187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.314998    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.315187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315008    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72548522-bba4-4f29-92ea-4d41d72e88b1-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.315187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315028    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.315187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315039    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.315187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315054    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.315187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315067    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.315187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315076    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.315187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315086    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.315187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315095    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7drp\" (UniqueName: \"kubernetes.io/projected/72548522-bba4-4f29-92ea-4d41d72e88b1-kube-api-access-j7drp\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.315187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315104    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.315187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315113    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.315187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315121    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/142d862b-ea80-4125-b065-7c3d471fe400-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.315187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315131    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.315187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315144    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72548522-bba4-4f29-92ea-4d41d72e88b1-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.315187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315155    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0378c741-36a6-4c68-97bb-f3c6f4ec3470-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.315187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315164    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/142d862b-ea80-4125-b065-7c3d471fe400-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.315187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315172    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0378c741-36a6-4c68-97bb-f3c6f4ec3470-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.315187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315180    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/036d7b93-15ba-499b-9ed5-a4217c856742-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.315187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315189    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kc9t\" (UniqueName: \"kubernetes.io/projected/036d7b93-15ba-499b-9ed5-a4217c856742-kube-api-access-8kc9t\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.315187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315198    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/036d7b93-15ba-499b-9ed5-a4217c856742-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.316266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315208    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kn7vn\" (UniqueName: \"kubernetes.io/projected/0378c741-36a6-4c68-97bb-f3c6f4ec3470-kube-api-access-kn7vn\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.316266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315216    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/036d7b93-15ba-499b-9ed5-a4217c856742-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.316266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315224    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9lrq\" (UniqueName: \"kubernetes.io/projected/142d862b-ea80-4125-b065-7c3d471fe400-kube-api-access-r9lrq\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.316266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315233    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/142d862b-ea80-4125-b065-7c3d471fe400-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.316266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315242    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/036d7b93-15ba-499b-9ed5-a4217c856742-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.316266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315505    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-workload-socket" pod "29ce830e-8096-4239-8ad4-03163e8fc5dc" (UID: "29ce830e-8096-4239-8ad4-03163e8fc5dc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.316266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.315700    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-credential-socket" pod "29ce830e-8096-4239-8ad4-03163e8fc5dc" (UID: "29ce830e-8096-4239-8ad4-03163e8fc5dc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.316266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.316156    5820 generic.go:356] "Generic (PLEG): container finished" podID="142d862b-ea80-4125-b065-7c3d471fe400" containerID="492e2c395aae3fa51c232eaa9bc21adb8734e3a721846da663dcb9cf8976f621" exitCode=0
Oct 30 19:00:22.316266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.316224    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8" event={"ID":"142d862b-ea80-4125-b065-7c3d471fe400","Type":"ContainerDied","Data":"492e2c395aae3fa51c232eaa9bc21adb8734e3a721846da663dcb9cf8976f621"}
Oct 30 19:00:22.316266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.316255    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8" event={"ID":"142d862b-ea80-4125-b065-7c3d471fe400","Type":"ContainerDied","Data":"7b6f3a67d4633fe0501a9b15f222a69c0f64747aa50d3a94fcd4c3c378a59063"}
Oct 30 19:00:22.316781 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.316310    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"
Oct 30 19:00:22.318806 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.318778    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-token" pod "29ce830e-8096-4239-8ad4-03163e8fc5dc" (UID: "29ce830e-8096-4239-8ad4-03163e8fc5dc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:22.319193 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.319168    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29ce830e-8096-4239-8ad4-03163e8fc5dc-istiod-ca-cert" pod "29ce830e-8096-4239-8ad4-03163e8fc5dc" (UID: "29ce830e-8096-4239-8ad4-03163e8fc5dc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:22.319468 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.319444    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-data" pod "29ce830e-8096-4239-8ad4-03163e8fc5dc" (UID: "29ce830e-8096-4239-8ad4-03163e8fc5dc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.319594 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.319565    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29ce830e-8096-4239-8ad4-03163e8fc5dc-kube-api-access-hjgwk" pod "29ce830e-8096-4239-8ad4-03163e8fc5dc" (UID: "29ce830e-8096-4239-8ad4-03163e8fc5dc"). InnerVolumeSpecName "kube-api-access-hjgwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:22.319843 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.319820    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-workload-certs" pod "29ce830e-8096-4239-8ad4-03163e8fc5dc" (UID: "29ce830e-8096-4239-8ad4-03163e8fc5dc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.327842 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.326025    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf" event={"ID":"a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c","Type":"ContainerDied","Data":"28c45efcb438f195a5f236f192ae2b3d99ab1713193f59e27ed2a3e704441283"}
Oct 30 19:00:22.327842 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.326130    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf"
Oct 30 19:00:22.328227 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.328068    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-podinfo" pod "29ce830e-8096-4239-8ad4-03163e8fc5dc" (UID: "29ce830e-8096-4239-8ad4-03163e8fc5dc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:00:22.329132 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.328852    5820 generic.go:356] "Generic (PLEG): container finished" podID="72548522-bba4-4f29-92ea-4d41d72e88b1" containerID="89dd0215184eaf3a324caddfa14a4745bfaa69895726f93d93c6d823e6269eea" exitCode=0
Oct 30 19:00:22.329132 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.328927    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs" event={"ID":"72548522-bba4-4f29-92ea-4d41d72e88b1","Type":"ContainerDied","Data":"89dd0215184eaf3a324caddfa14a4745bfaa69895726f93d93c6d823e6269eea"}
Oct 30 19:00:22.329266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.329134    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"
Oct 30 19:00:22.329404 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.329363    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs" event={"ID":"72548522-bba4-4f29-92ea-4d41d72e88b1","Type":"ContainerDied","Data":"cf4c6fcc438037db85d7f7d66a37af5f0aed09fdae2d4c41ac4ce11d070314b9"}
Oct 30 19:00:22.333379 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.333352    5820 generic.go:356] "Generic (PLEG): container finished" podID="29ce830e-8096-4239-8ad4-03163e8fc5dc" containerID="def293b92810a87ddde97b30ad90ddcff6ebda969bf0871b9dda302d9c74cf69" exitCode=0
Oct 30 19:00:22.333768 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.333512    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796" event={"ID":"29ce830e-8096-4239-8ad4-03163e8fc5dc","Type":"ContainerDied","Data":"def293b92810a87ddde97b30ad90ddcff6ebda969bf0871b9dda302d9c74cf69"}
Oct 30 19:00:22.333768 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.333554    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796" event={"ID":"29ce830e-8096-4239-8ad4-03163e8fc5dc","Type":"ContainerDied","Data":"5157de500baa93d526308e8dc72e047d7091f23bb230a8c0e54517828452446a"}
Oct 30 19:00:22.334099 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.334068    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"
Oct 30 19:00:22.334188 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.334154    5820 scope.go:122] "RemoveContainer" containerID="56de44c8af394d2e1b66452336dfa3f45613827ca0ff184be4f5a080bf487366"
Oct 30 19:00:22.334688 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:00:22.334659    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56de44c8af394d2e1b66452336dfa3f45613827ca0ff184be4f5a080bf487366\": container with ID starting with 56de44c8af394d2e1b66452336dfa3f45613827ca0ff184be4f5a080bf487366 not found: ID does not exist" containerID="56de44c8af394d2e1b66452336dfa3f45613827ca0ff184be4f5a080bf487366"
Oct 30 19:00:22.334825 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.334701    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56de44c8af394d2e1b66452336dfa3f45613827ca0ff184be4f5a080bf487366"} err="failed to get container status \"56de44c8af394d2e1b66452336dfa3f45613827ca0ff184be4f5a080bf487366\": rpc error: code = NotFound desc = could not find container \"56de44c8af394d2e1b66452336dfa3f45613827ca0ff184be4f5a080bf487366\": container with ID starting with 56de44c8af394d2e1b66452336dfa3f45613827ca0ff184be4f5a080bf487366 not found: ID does not exist"
Oct 30 19:00:22.334825 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.334740    5820 scope.go:122] "RemoveContainer" containerID="492e2c395aae3fa51c232eaa9bc21adb8734e3a721846da663dcb9cf8976f621"
Oct 30 19:00:22.335394 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.335368    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-envoy" pod "29ce830e-8096-4239-8ad4-03163e8fc5dc" (UID: "29ce830e-8096-4239-8ad4-03163e8fc5dc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.336035 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.336011    5820 generic.go:356] "Generic (PLEG): container finished" podID="0378c741-36a6-4c68-97bb-f3c6f4ec3470" containerID="4e0cd1fe656371a287b55fd9340a0b73a1b8faa9570caf654416b39c35bac92c" exitCode=0
Oct 30 19:00:22.336143 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.336051    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc" event={"ID":"0378c741-36a6-4c68-97bb-f3c6f4ec3470","Type":"ContainerDied","Data":"4e0cd1fe656371a287b55fd9340a0b73a1b8faa9570caf654416b39c35bac92c"}
Oct 30 19:00:22.336143 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.336086    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc" event={"ID":"0378c741-36a6-4c68-97bb-f3c6f4ec3470","Type":"ContainerDied","Data":"48e8e9968c3c8dc0dfda3c214f3988067e36bed1f66ddb1c1ad1279fdae5987b"}
Oct 30 19:00:22.336268 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.336152    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"
Oct 30 19:00:22.348438 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.348417    5820 scope.go:122] "RemoveContainer" containerID="492e2c395aae3fa51c232eaa9bc21adb8734e3a721846da663dcb9cf8976f621"
Oct 30 19:00:22.348737 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:00:22.348719    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"492e2c395aae3fa51c232eaa9bc21adb8734e3a721846da663dcb9cf8976f621\": container with ID starting with 492e2c395aae3fa51c232eaa9bc21adb8734e3a721846da663dcb9cf8976f621 not found: ID does not exist" containerID="492e2c395aae3fa51c232eaa9bc21adb8734e3a721846da663dcb9cf8976f621"
Oct 30 19:00:22.348814 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.348746    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"492e2c395aae3fa51c232eaa9bc21adb8734e3a721846da663dcb9cf8976f621"} err="failed to get container status \"492e2c395aae3fa51c232eaa9bc21adb8734e3a721846da663dcb9cf8976f621\": rpc error: code = NotFound desc = could not find container \"492e2c395aae3fa51c232eaa9bc21adb8734e3a721846da663dcb9cf8976f621\": container with ID starting with 492e2c395aae3fa51c232eaa9bc21adb8734e3a721846da663dcb9cf8976f621 not found: ID does not exist"
Oct 30 19:00:22.348814 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.348766    5820 scope.go:122] "RemoveContainer" containerID="bde1cc902caac9e5de579bfa6fc2873b70ef697f39faa14202e89f189d9fc08b"
Oct 30 19:00:22.357515 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.357495    5820 scope.go:122] "RemoveContainer" containerID="89dd0215184eaf3a324caddfa14a4745bfaa69895726f93d93c6d823e6269eea"
Oct 30 19:00:22.366438 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.366418    5820 scope.go:122] "RemoveContainer" containerID="89dd0215184eaf3a324caddfa14a4745bfaa69895726f93d93c6d823e6269eea"
Oct 30 19:00:22.366694 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:00:22.366676    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89dd0215184eaf3a324caddfa14a4745bfaa69895726f93d93c6d823e6269eea\": container with ID starting with 89dd0215184eaf3a324caddfa14a4745bfaa69895726f93d93c6d823e6269eea not found: ID does not exist" containerID="89dd0215184eaf3a324caddfa14a4745bfaa69895726f93d93c6d823e6269eea"
Oct 30 19:00:22.366736 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.366702    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89dd0215184eaf3a324caddfa14a4745bfaa69895726f93d93c6d823e6269eea"} err="failed to get container status \"89dd0215184eaf3a324caddfa14a4745bfaa69895726f93d93c6d823e6269eea\": rpc error: code = NotFound desc = could not find container \"89dd0215184eaf3a324caddfa14a4745bfaa69895726f93d93c6d823e6269eea\": container with ID starting with 89dd0215184eaf3a324caddfa14a4745bfaa69895726f93d93c6d823e6269eea not found: ID does not exist"
Oct 30 19:00:22.366736 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.366719    5820 scope.go:122] "RemoveContainer" containerID="def293b92810a87ddde97b30ad90ddcff6ebda969bf0871b9dda302d9c74cf69"
Oct 30 19:00:22.375698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.375678    5820 scope.go:122] "RemoveContainer" containerID="def293b92810a87ddde97b30ad90ddcff6ebda969bf0871b9dda302d9c74cf69"
Oct 30 19:00:22.376004 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:00:22.375983    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"def293b92810a87ddde97b30ad90ddcff6ebda969bf0871b9dda302d9c74cf69\": container with ID starting with def293b92810a87ddde97b30ad90ddcff6ebda969bf0871b9dda302d9c74cf69 not found: ID does not exist" containerID="def293b92810a87ddde97b30ad90ddcff6ebda969bf0871b9dda302d9c74cf69"
Oct 30 19:00:22.376077 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.376010    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"def293b92810a87ddde97b30ad90ddcff6ebda969bf0871b9dda302d9c74cf69"} err="failed to get container status \"def293b92810a87ddde97b30ad90ddcff6ebda969bf0871b9dda302d9c74cf69\": rpc error: code = NotFound desc = could not find container \"def293b92810a87ddde97b30ad90ddcff6ebda969bf0871b9dda302d9c74cf69\": container with ID starting with def293b92810a87ddde97b30ad90ddcff6ebda969bf0871b9dda302d9c74cf69 not found: ID does not exist"
Oct 30 19:00:22.376077 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.376025    5820 scope.go:122] "RemoveContainer" containerID="4e0cd1fe656371a287b55fd9340a0b73a1b8faa9570caf654416b39c35bac92c"
Oct 30 19:00:22.384655 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.384636    5820 scope.go:122] "RemoveContainer" containerID="4e0cd1fe656371a287b55fd9340a0b73a1b8faa9570caf654416b39c35bac92c"
Oct 30 19:00:22.384895 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:00:22.384878    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e0cd1fe656371a287b55fd9340a0b73a1b8faa9570caf654416b39c35bac92c\": container with ID starting with 4e0cd1fe656371a287b55fd9340a0b73a1b8faa9570caf654416b39c35bac92c not found: ID does not exist" containerID="4e0cd1fe656371a287b55fd9340a0b73a1b8faa9570caf654416b39c35bac92c"
Oct 30 19:00:22.384937 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.384902    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e0cd1fe656371a287b55fd9340a0b73a1b8faa9570caf654416b39c35bac92c"} err="failed to get container status \"4e0cd1fe656371a287b55fd9340a0b73a1b8faa9570caf654416b39c35bac92c\": rpc error: code = NotFound desc = could not find container \"4e0cd1fe656371a287b55fd9340a0b73a1b8faa9570caf654416b39c35bac92c\": container with ID starting with 4e0cd1fe656371a287b55fd9340a0b73a1b8faa9570caf654416b39c35bac92c not found: ID does not exist"
Oct 30 19:00:22.416943 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.416911    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjgwk\" (UniqueName: \"kubernetes.io/projected/29ce830e-8096-4239-8ad4-03163e8fc5dc-kube-api-access-hjgwk\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.417117 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.416942    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.417117 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.416999    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.417117 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.417008    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.417117 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.417017    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29ce830e-8096-4239-8ad4-03163e8fc5dc-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.417117 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.417025    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.417117 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.417033    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.417117 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.417048    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ce830e-8096-4239-8ad4-03163e8fc5dc-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.417117 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.417059    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29ce830e-8096-4239-8ad4-03163e8fc5dc-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.700303 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.700242    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"]
Oct 30 19:00:22.707527 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.707452    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-2ngd8"]
Oct 30 19:00:22.825352 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.825316    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"]
Oct 30 19:00:22.841569 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.841534    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9xwzw"]
Oct 30 19:00:22.951506 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.951429    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf"]
Oct 30 19:00:22.954389 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:22.954367    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-470093-29889780-nmmdf"]
Oct 30 19:00:23.065911 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:23.065872    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"]
Oct 30 19:00:23.068927 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:23.068901    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-hmprs"]
Oct 30 19:00:23.181653 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:23.181607    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"]
Oct 30 19:00:23.184441 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:23.184406    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-nfbqc"]
Oct 30 19:00:23.458031 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:23.457986    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="036d7b93-15ba-499b-9ed5-a4217c856742" path="/var/lib/kubelet/pods/036d7b93-15ba-499b-9ed5-a4217c856742/volumes"
Oct 30 19:00:23.458541 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:23.458520    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0378c741-36a6-4c68-97bb-f3c6f4ec3470" path="/var/lib/kubelet/pods/0378c741-36a6-4c68-97bb-f3c6f4ec3470/volumes"
Oct 30 19:00:23.459043 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:23.459026    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="142d862b-ea80-4125-b065-7c3d471fe400" path="/var/lib/kubelet/pods/142d862b-ea80-4125-b065-7c3d471fe400/volumes"
Oct 30 19:00:23.459819 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:23.459799    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="72548522-bba4-4f29-92ea-4d41d72e88b1" path="/var/lib/kubelet/pods/72548522-bba4-4f29-92ea-4d41d72e88b1/volumes"
Oct 30 19:00:23.460255 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:23.460236    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c" path="/var/lib/kubelet/pods/a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c/volumes"
Oct 30 19:00:23.460624 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:23.460606    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"]
Oct 30 19:00:23.460698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:23.460647    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cf796"]
Oct 30 19:00:23.965375 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:23.959246    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:25.437797 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:25.437760    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29ce830e-8096-4239-8ad4-03163e8fc5dc" path="/var/lib/kubelet/pods/29ce830e-8096-4239-8ad4-03163e8fc5dc/volumes"
Oct 30 19:00:28.569630 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.569597    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"]
Oct 30 19:00:28.570274 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570257    5820 cpu_manager.go:414] "containerMap: removing container" podUID="a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c" containerName="cleanup"
Oct 30 19:00:28.570347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570276    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c" containerName="cleanup"
Oct 30 19:00:28.570347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570290    5820 cpu_manager.go:414] "containerMap: removing container" podUID="72548522-bba4-4f29-92ea-4d41d72e88b1" containerName="istio-proxy"
Oct 30 19:00:28.570347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570295    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="72548522-bba4-4f29-92ea-4d41d72e88b1" containerName="istio-proxy"
Oct 30 19:00:28.570347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570301    5820 cpu_manager.go:414] "containerMap: removing container" podUID="0378c741-36a6-4c68-97bb-f3c6f4ec3470" containerName="istio-proxy"
Oct 30 19:00:28.570347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570306    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0378c741-36a6-4c68-97bb-f3c6f4ec3470" containerName="istio-proxy"
Oct 30 19:00:28.570347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570316    5820 cpu_manager.go:414] "containerMap: removing container" podUID="a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c" containerName="cleanup"
Oct 30 19:00:28.570347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570321    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c" containerName="cleanup"
Oct 30 19:00:28.570347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570328    5820 cpu_manager.go:414] "containerMap: removing container" podUID="036d7b93-15ba-499b-9ed5-a4217c856742" containerName="istio-proxy"
Oct 30 19:00:28.570347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570332    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="036d7b93-15ba-499b-9ed5-a4217c856742" containerName="istio-proxy"
Oct 30 19:00:28.570347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570345    5820 cpu_manager.go:414] "containerMap: removing container" podUID="142d862b-ea80-4125-b065-7c3d471fe400" containerName="istio-proxy"
Oct 30 19:00:28.570347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570349    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="142d862b-ea80-4125-b065-7c3d471fe400" containerName="istio-proxy"
Oct 30 19:00:28.570698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570368    5820 cpu_manager.go:414] "containerMap: removing container" podUID="a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c" containerName="cleanup"
Oct 30 19:00:28.570698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570372    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c" containerName="cleanup"
Oct 30 19:00:28.570698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570380    5820 cpu_manager.go:414] "containerMap: removing container" podUID="180fc1d4-d421-446a-afd7-f1f59bfafe0f" containerName="istio-proxy"
Oct 30 19:00:28.570698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570384    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="180fc1d4-d421-446a-afd7-f1f59bfafe0f" containerName="istio-proxy"
Oct 30 19:00:28.570698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570391    5820 cpu_manager.go:414] "containerMap: removing container" podUID="1057f570-7bb3-4bfb-ab81-6696e3b792b2" containerName="istio-proxy"
Oct 30 19:00:28.570698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570394    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1057f570-7bb3-4bfb-ab81-6696e3b792b2" containerName="istio-proxy"
Oct 30 19:00:28.570698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570415    5820 cpu_manager.go:414] "containerMap: removing container" podUID="29ce830e-8096-4239-8ad4-03163e8fc5dc" containerName="istio-proxy"
Oct 30 19:00:28.570698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570419    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29ce830e-8096-4239-8ad4-03163e8fc5dc" containerName="istio-proxy"
Oct 30 19:00:28.570698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570500    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="142d862b-ea80-4125-b065-7c3d471fe400" containerName="istio-proxy"
Oct 30 19:00:28.570698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570516    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c" containerName="cleanup"
Oct 30 19:00:28.570698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570523    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="180fc1d4-d421-446a-afd7-f1f59bfafe0f" containerName="istio-proxy"
Oct 30 19:00:28.570698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570528    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="1057f570-7bb3-4bfb-ab81-6696e3b792b2" containerName="istio-proxy"
Oct 30 19:00:28.570698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570537    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="29ce830e-8096-4239-8ad4-03163e8fc5dc" containerName="istio-proxy"
Oct 30 19:00:28.570698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570549    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="72548522-bba4-4f29-92ea-4d41d72e88b1" containerName="istio-proxy"
Oct 30 19:00:28.570698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570556    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="0378c741-36a6-4c68-97bb-f3c6f4ec3470" containerName="istio-proxy"
Oct 30 19:00:28.570698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570564    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c" containerName="cleanup"
Oct 30 19:00:28.570698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.570572    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="036d7b93-15ba-499b-9ed5-a4217c856742" containerName="istio-proxy"
Oct 30 19:00:28.574356 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.574337    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"
Oct 30 19:00:28.631437 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.631413    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-66baf45c-serving-cert\""
Oct 30 19:00:28.686040 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.686006    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"]
Oct 30 19:00:28.739659 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.739625    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-926m8\" (UniqueName: \"kubernetes.io/projected/7d7e45b3-4b0d-4789-b6be-17f080656583-kube-api-access-926m8\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr\" (UID: \"7d7e45b3-4b0d-4789-b6be-17f080656583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"
Oct 30 19:00:28.739802 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.739679    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d7e45b3-4b0d-4789-b6be-17f080656583-openshift-service-ca\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr\" (UID: \"7d7e45b3-4b0d-4789-b6be-17f080656583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"
Oct 30 19:00:28.739802 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.739784    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d7e45b3-4b0d-4789-b6be-17f080656583-metrics-certs\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr\" (UID: \"7d7e45b3-4b0d-4789-b6be-17f080656583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"
Oct 30 19:00:28.739895 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.739871    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d7e45b3-4b0d-4789-b6be-17f080656583-maas-api-tls\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr\" (UID: \"7d7e45b3-4b0d-4789-b6be-17f080656583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"
Oct 30 19:00:28.839148 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.839051    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:28.844735 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.844678    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d7e45b3-4b0d-4789-b6be-17f080656583-maas-api-tls\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr\" (UID: \"7d7e45b3-4b0d-4789-b6be-17f080656583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"
Oct 30 19:00:28.846244 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.846179    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-926m8\" (UniqueName: \"kubernetes.io/projected/7d7e45b3-4b0d-4789-b6be-17f080656583-kube-api-access-926m8\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr\" (UID: \"7d7e45b3-4b0d-4789-b6be-17f080656583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"
Oct 30 19:00:28.846878 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.846858    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d7e45b3-4b0d-4789-b6be-17f080656583-openshift-service-ca\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr\" (UID: \"7d7e45b3-4b0d-4789-b6be-17f080656583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"
Oct 30 19:00:28.848077 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:00:28.845284    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-serving-cert: secret "maas-api-e2e-mt-66baf45c-serving-cert" not found
Oct 30 19:00:28.848364 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:00:28.848351    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d7e45b3-4b0d-4789-b6be-17f080656583-maas-api-tls podName:7d7e45b3-4b0d-4789-b6be-17f080656583 nodeName:}" failed. No retries permitted until 2026-10-30 19:00:29.348328391 +0000 UTC m=+1550.060077083 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7d7e45b3-4b0d-4789-b6be-17f080656583-maas-api-tls") pod "maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr" (UID: "7d7e45b3-4b0d-4789-b6be-17f080656583") : secret "maas-api-e2e-mt-66baf45c-serving-cert" not found
Oct 30 19:00:28.848519 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.847879    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d7e45b3-4b0d-4789-b6be-17f080656583-openshift-service-ca\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr\" (UID: \"7d7e45b3-4b0d-4789-b6be-17f080656583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"
Oct 30 19:00:28.849055 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.849015    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d7e45b3-4b0d-4789-b6be-17f080656583-metrics-certs\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr\" (UID: \"7d7e45b3-4b0d-4789-b6be-17f080656583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"
Oct 30 19:00:28.873329 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.873298    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d7e45b3-4b0d-4789-b6be-17f080656583-metrics-certs\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr\" (UID: \"7d7e45b3-4b0d-4789-b6be-17f080656583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"
Oct 30 19:00:28.920913 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:28.920886    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-926m8\" (UniqueName: \"kubernetes.io/projected/7d7e45b3-4b0d-4789-b6be-17f080656583-kube-api-access-926m8\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr\" (UID: \"7d7e45b3-4b0d-4789-b6be-17f080656583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"
Oct 30 19:00:29.039129 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:29.039090    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:29.358749 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:29.358712    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d7e45b3-4b0d-4789-b6be-17f080656583-maas-api-tls\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr\" (UID: \"7d7e45b3-4b0d-4789-b6be-17f080656583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"
Oct 30 19:00:29.361359 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:29.361325    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d7e45b3-4b0d-4789-b6be-17f080656583-maas-api-tls\") pod \"maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr\" (UID: \"7d7e45b3-4b0d-4789-b6be-17f080656583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"
Oct 30 19:00:29.385622 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:29.385590    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:29.487667 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:29.487621    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"
Oct 30 19:00:29.873347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:29.873320    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"]
Oct 30 19:00:29.874424 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:00:29.874398    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d7e45b3_4b0d_4789_b6be_17f080656583.slice/crio-aa4afeeb4c614765a04959eef67574649c0fff1f74c0bdb4014fb09e577072dd WatchSource:0}: Error finding container aa4afeeb4c614765a04959eef67574649c0fff1f74c0bdb4014fb09e577072dd: Status 404 returned error can't find the container with id aa4afeeb4c614765a04959eef67574649c0fff1f74c0bdb4014fb09e577072dd
Oct 30 19:00:29.875845 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:29.875828    5820 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:00:30.399648 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:30.399617    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr" event={"ID":"7d7e45b3-4b0d-4789-b6be-17f080656583","Type":"ContainerStarted","Data":"aa4afeeb4c614765a04959eef67574649c0fff1f74c0bdb4014fb09e577072dd"}
Oct 30 19:00:31.408204 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:31.408165    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr" event={"ID":"7d7e45b3-4b0d-4789-b6be-17f080656583","Type":"ContainerStarted","Data":"022ae211234b0ed1e28b2d97caf594b378e88339a2b9bd15625906e6b9a36501"}
Oct 30 19:00:31.408619 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:31.408341    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"
Oct 30 19:00:31.533868 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:31.533821    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr" podStartSLOduration=2.500570137 podStartE2EDuration="3.53380617s" totalImagesPullingTime="1.033236033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:31.530757602 +0000 UTC m=+1552.242506298" watchObservedRunningTime="2026-10-30 19:00:31.53380617 +0000 UTC m=+1552.245554866"
Oct 30 19:00:34.060976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:34.060072    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:37.420504 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:37.420476    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"
Oct 30 19:00:38.204750 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:00:38.204708    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:00:38.838629 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:38.838588    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"]
Oct 30 19:00:38.839580 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:38.839557    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="a36e9cac-95e2-4e7e-8c7a-fab0e6c39d5c" containerName="cleanup"
Oct 30 19:00:38.843245 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:38.843224    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"
Oct 30 19:00:38.898865 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:38.898796    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b3120c6-159a-401d-b8ea-bc03ebbd056e-openshift-service-ca\") pod \"maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m\" (UID: \"5b3120c6-159a-401d-b8ea-bc03ebbd056e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"
Oct 30 19:00:38.899097 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:38.899059    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b3120c6-159a-401d-b8ea-bc03ebbd056e-metrics-certs\") pod \"maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m\" (UID: \"5b3120c6-159a-401d-b8ea-bc03ebbd056e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"
Oct 30 19:00:38.899189 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:38.899167    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnb2m\" (UniqueName: \"kubernetes.io/projected/5b3120c6-159a-401d-b8ea-bc03ebbd056e-kube-api-access-gnb2m\") pod \"maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m\" (UID: \"5b3120c6-159a-401d-b8ea-bc03ebbd056e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"
Oct 30 19:00:38.899252 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:38.899205    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b3120c6-159a-401d-b8ea-bc03ebbd056e-maas-api-tls\") pod \"maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m\" (UID: \"5b3120c6-159a-401d-b8ea-bc03ebbd056e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"
Oct 30 19:00:38.947358 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:38.947316    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-4ebcd2-serving-cert\""
Oct 30 19:00:38.950674 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:38.950646    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"]
Oct 30 19:00:39.000456 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:39.000417    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b3120c6-159a-401d-b8ea-bc03ebbd056e-openshift-service-ca\") pod \"maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m\" (UID: \"5b3120c6-159a-401d-b8ea-bc03ebbd056e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"
Oct 30 19:00:39.000615 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:39.000519    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b3120c6-159a-401d-b8ea-bc03ebbd056e-metrics-certs\") pod \"maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m\" (UID: \"5b3120c6-159a-401d-b8ea-bc03ebbd056e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"
Oct 30 19:00:39.000615 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:39.000570    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnb2m\" (UniqueName: \"kubernetes.io/projected/5b3120c6-159a-401d-b8ea-bc03ebbd056e-kube-api-access-gnb2m\") pod \"maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m\" (UID: \"5b3120c6-159a-401d-b8ea-bc03ebbd056e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"
Oct 30 19:00:39.000615 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:39.000603    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b3120c6-159a-401d-b8ea-bc03ebbd056e-maas-api-tls\") pod \"maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m\" (UID: \"5b3120c6-159a-401d-b8ea-bc03ebbd056e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"
Oct 30 19:00:39.000815 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:00:39.000793    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-serving-cert: secret "maas-api-e2e-api-b-4ebcd2-serving-cert" not found
Oct 30 19:00:39.000889 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:00:39.000877    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b3120c6-159a-401d-b8ea-bc03ebbd056e-maas-api-tls podName:5b3120c6-159a-401d-b8ea-bc03ebbd056e nodeName:}" failed. No retries permitted until 2026-10-30 19:00:39.500856883 +0000 UTC m=+1560.212605576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5b3120c6-159a-401d-b8ea-bc03ebbd056e-maas-api-tls") pod "maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m" (UID: "5b3120c6-159a-401d-b8ea-bc03ebbd056e") : secret "maas-api-e2e-api-b-4ebcd2-serving-cert" not found
Oct 30 19:00:39.001205 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:39.001183    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b3120c6-159a-401d-b8ea-bc03ebbd056e-openshift-service-ca\") pod \"maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m\" (UID: \"5b3120c6-159a-401d-b8ea-bc03ebbd056e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"
Oct 30 19:00:39.003176 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:39.003160    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b3120c6-159a-401d-b8ea-bc03ebbd056e-metrics-certs\") pod \"maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m\" (UID: \"5b3120c6-159a-401d-b8ea-bc03ebbd056e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"
Oct 30 19:00:39.066300 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:39.066271    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnb2m\" (UniqueName: \"kubernetes.io/projected/5b3120c6-159a-401d-b8ea-bc03ebbd056e-kube-api-access-gnb2m\") pod \"maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m\" (UID: \"5b3120c6-159a-401d-b8ea-bc03ebbd056e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"
Oct 30 19:00:39.107577 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:39.107474    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:39.300010 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:39.299964    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:39.517570 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:39.517464    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b3120c6-159a-401d-b8ea-bc03ebbd056e-maas-api-tls\") pod \"maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m\" (UID: \"5b3120c6-159a-401d-b8ea-bc03ebbd056e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"
Oct 30 19:00:39.519940 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:39.519919    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b3120c6-159a-401d-b8ea-bc03ebbd056e-maas-api-tls\") pod \"maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m\" (UID: \"5b3120c6-159a-401d-b8ea-bc03ebbd056e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"
Oct 30 19:00:39.525239 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:39.525217    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:39.745488 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:39.745459    5820 scope.go:122] "RemoveContainer" containerID="9ee0872e94052e046652ebecab0c991b075fc7ab7a2e18814f9eea958d629189"
Oct 30 19:00:39.757459 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:39.757434    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"
Oct 30 19:00:40.157688 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:40.157660    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"]
Oct 30 19:00:40.163967 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:00:40.163906    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b3120c6_159a_401d_b8ea_bc03ebbd056e.slice/crio-5b4aa03e758b72d49e7acf9b4533fea0d4d5ab8b3c91944a222281191c8286e9 WatchSource:0}: Error finding container 5b4aa03e758b72d49e7acf9b4533fea0d4d5ab8b3c91944a222281191c8286e9: Status 404 returned error can't find the container with id 5b4aa03e758b72d49e7acf9b4533fea0d4d5ab8b3c91944a222281191c8286e9
Oct 30 19:00:40.471230 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:40.471149    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m" event={"ID":"5b3120c6-159a-401d-b8ea-bc03ebbd056e","Type":"ContainerStarted","Data":"5b4aa03e758b72d49e7acf9b4533fea0d4d5ab8b3c91944a222281191c8286e9"}
Oct 30 19:00:41.479432 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:41.479384    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m" event={"ID":"5b3120c6-159a-401d-b8ea-bc03ebbd056e","Type":"ContainerStarted","Data":"c145265cba191e358f914a6f5c550815119b3b2ccd268571260c791a8fa249a1"}
Oct 30 19:00:41.479909 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:41.479548    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"
Oct 30 19:00:41.602509 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:41.602451    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m" podStartSLOduration=2.7533902660000003 podStartE2EDuration="3.602434034s" totalImagesPullingTime="849.043768ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:41.601190101 +0000 UTC m=+1562.312938798" watchObservedRunningTime="2026-10-30 19:00:41.602434034 +0000 UTC m=+1562.314182731"
Oct 30 19:00:44.258012 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:44.256579    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:47.493900 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:47.493865    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"
Oct 30 19:00:47.727090 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:47.727051    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"]
Oct 30 19:00:47.727369 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:47.727312    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b" podUID="cfb02c85-4568-4d67-9dfa-2c5252c3a67b" containerName="istio-proxy" containerID="cri-o://d84e87d80cf51fa0b067b2f4394d8eed3857a5ca1db7e334cf42b0103db7b955" gracePeriod=30
Oct 30 19:00:48.001333 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:48.001238    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:49.872602 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:49.872061    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:53.008454 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.008395    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 19:00:53.159146 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.159112    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-workload-socket\") pod \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") "
Oct 30 19:00:53.159319 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.159168    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-kube-api-access-grhq5\" (UniqueName: \"kubernetes.io/projected/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-kube-api-access-grhq5\") pod \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") "
Oct 30 19:00:53.159319 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.159190    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-credential-socket\") pod \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") "
Oct 30 19:00:53.159319 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.159222    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-workload-certs\") pod \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") "
Oct 30 19:00:53.159319 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.159262    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-envoy\") pod \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") "
Oct 30 19:00:53.159319 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.159291    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-podinfo\") pod \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") "
Oct 30 19:00:53.159584 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.159338    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istiod-ca-cert\") pod \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") "
Oct 30 19:00:53.159584 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.159365    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-token\" (UniqueName: \"kubernetes.io/projected/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-token\") pod \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") "
Oct 30 19:00:53.159584 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.159422    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-workload-socket" pod "cfb02c85-4568-4d67-9dfa-2c5252c3a67b" (UID: "cfb02c85-4568-4d67-9dfa-2c5252c3a67b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:53.159584 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.159545    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-credential-socket" pod "cfb02c85-4568-4d67-9dfa-2c5252c3a67b" (UID: "cfb02c85-4568-4d67-9dfa-2c5252c3a67b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:53.159584 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.159554    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-workload-certs" pod "cfb02c85-4568-4d67-9dfa-2c5252c3a67b" (UID: "cfb02c85-4568-4d67-9dfa-2c5252c3a67b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:53.159584 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.159581    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-data\") pod \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\" (UID: \"cfb02c85-4568-4d67-9dfa-2c5252c3a67b\") "
Oct 30 19:00:53.159895 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.159782    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istiod-ca-cert" pod "cfb02c85-4568-4d67-9dfa-2c5252c3a67b" (UID: "cfb02c85-4568-4d67-9dfa-2c5252c3a67b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:53.159995 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.159889    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-data" pod "cfb02c85-4568-4d67-9dfa-2c5252c3a67b" (UID: "cfb02c85-4568-4d67-9dfa-2c5252c3a67b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:53.160259 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.160241    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:53.160320 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.160260    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:53.160320 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.160271    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:53.160320 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.160280    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:53.160320 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.160290    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:53.161645 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.161622    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-kube-api-access-grhq5" pod "cfb02c85-4568-4d67-9dfa-2c5252c3a67b" (UID: "cfb02c85-4568-4d67-9dfa-2c5252c3a67b"). InnerVolumeSpecName "kube-api-access-grhq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:53.161750 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.161732    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-podinfo" pod "cfb02c85-4568-4d67-9dfa-2c5252c3a67b" (UID: "cfb02c85-4568-4d67-9dfa-2c5252c3a67b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:00:53.161812 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.161797    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-token" pod "cfb02c85-4568-4d67-9dfa-2c5252c3a67b" (UID: "cfb02c85-4568-4d67-9dfa-2c5252c3a67b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:53.162203 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.162187    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-envoy" pod "cfb02c85-4568-4d67-9dfa-2c5252c3a67b" (UID: "cfb02c85-4568-4d67-9dfa-2c5252c3a67b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:53.262257 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.262178    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grhq5\" (UniqueName: \"kubernetes.io/projected/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-kube-api-access-grhq5\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:53.262257 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.262207    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:53.262257 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.262217    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:53.262257 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.262225    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfb02c85-4568-4d67-9dfa-2c5252c3a67b-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:53.606348 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.606310    5820 generic.go:356] "Generic (PLEG): container finished" podID="cfb02c85-4568-4d67-9dfa-2c5252c3a67b" containerID="d84e87d80cf51fa0b067b2f4394d8eed3857a5ca1db7e334cf42b0103db7b955" exitCode=0
Oct 30 19:00:53.606528 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.606355    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"
Oct 30 19:00:53.606528 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.606396    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b" event={"ID":"cfb02c85-4568-4d67-9dfa-2c5252c3a67b","Type":"ContainerDied","Data":"d84e87d80cf51fa0b067b2f4394d8eed3857a5ca1db7e334cf42b0103db7b955"}
Oct 30 19:00:53.606528 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.606459    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b" event={"ID":"cfb02c85-4568-4d67-9dfa-2c5252c3a67b","Type":"ContainerDied","Data":"e2232e317dbfb69f2e6c6b2e6e6753fae4d1a792072b94ac8ae360b3f198dbcf"}
Oct 30 19:00:53.606528 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.606482    5820 scope.go:122] "RemoveContainer" containerID="d84e87d80cf51fa0b067b2f4394d8eed3857a5ca1db7e334cf42b0103db7b955"
Oct 30 19:00:53.616595 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.616576    5820 scope.go:122] "RemoveContainer" containerID="d84e87d80cf51fa0b067b2f4394d8eed3857a5ca1db7e334cf42b0103db7b955"
Oct 30 19:00:53.616870 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:00:53.616852    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d84e87d80cf51fa0b067b2f4394d8eed3857a5ca1db7e334cf42b0103db7b955\": container with ID starting with d84e87d80cf51fa0b067b2f4394d8eed3857a5ca1db7e334cf42b0103db7b955 not found: ID does not exist" containerID="d84e87d80cf51fa0b067b2f4394d8eed3857a5ca1db7e334cf42b0103db7b955"
Oct 30 19:00:53.616919 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.616879    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d84e87d80cf51fa0b067b2f4394d8eed3857a5ca1db7e334cf42b0103db7b955"} err="failed to get container status \"d84e87d80cf51fa0b067b2f4394d8eed3857a5ca1db7e334cf42b0103db7b955\": rpc error: code = NotFound desc = could not find container \"d84e87d80cf51fa0b067b2f4394d8eed3857a5ca1db7e334cf42b0103db7b955\": container with ID starting with d84e87d80cf51fa0b067b2f4394d8eed3857a5ca1db7e334cf42b0103db7b955 not found: ID does not exist"
Oct 30 19:00:53.779222 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.779195    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"]
Oct 30 19:00:53.828200 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:53.828165    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-2zg2b"]
Oct 30 19:00:55.088720 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.088680    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:55.285659 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.285622    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:55.438143 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.438064    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cfb02c85-4568-4d67-9dfa-2c5252c3a67b" path="/var/lib/kubelet/pods/cfb02c85-4568-4d67-9dfa-2c5252c3a67b/volumes"
Oct 30 19:00:55.520726 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.520685    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:55.659700 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.659669    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"]
Oct 30 19:00:55.660407 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.660390    5820 cpu_manager.go:414] "containerMap: removing container" podUID="cfb02c85-4568-4d67-9dfa-2c5252c3a67b" containerName="istio-proxy"
Oct 30 19:00:55.660484 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.660408    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfb02c85-4568-4d67-9dfa-2c5252c3a67b" containerName="istio-proxy"
Oct 30 19:00:55.660522 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.660496    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfb02c85-4568-4d67-9dfa-2c5252c3a67b" containerName="istio-proxy"
Oct 30 19:00:55.665371 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.665353    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.724461 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.724386    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-66baf45c-openshift-default-dockercfg-sqvln\""
Oct 30 19:00:55.779097 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.779067    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"]
Oct 30 19:00:55.811198 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.811167    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-envoy\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.811406 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.811222    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-data\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.811406 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.811345    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-workload-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.811406 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.811386    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-workload-certs\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.811406 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.811401    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istiod-ca-cert\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.811646 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.811466    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4zhg\" (UniqueName: \"kubernetes.io/projected/1aac21cb-584a-42ad-9dc6-99aada4cdba1-kube-api-access-c4zhg\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.811646 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.811491    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-credential-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.811646 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.811512    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-podinfo\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.811646 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.811530    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-token\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.913317 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.913276    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-workload-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.913516 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.913337    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-workload-certs\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.913516 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.913353    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istiod-ca-cert\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.913516 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.913427    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4zhg\" (UniqueName: \"kubernetes.io/projected/1aac21cb-584a-42ad-9dc6-99aada4cdba1-kube-api-access-c4zhg\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.913516 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.913470    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-credential-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.913516 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.913499    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-podinfo\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.914176 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.913522    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-token\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.914176 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.913641    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-envoy\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.914176 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.913724    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-data\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.914176 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.913936    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-credential-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.914392 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.914264    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-workload-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.914392 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.914317    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-workload-certs\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.914490 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.914455    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istiod-ca-cert\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.915017 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.914995    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-data\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.917109 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.917072    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-podinfo\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.917472 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.917449    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-envoy\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:55.994114 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:55.994040    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4zhg\" (UniqueName: \"kubernetes.io/projected/1aac21cb-584a-42ad-9dc6-99aada4cdba1-kube-api-access-c4zhg\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:56.000678 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:56.000657    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-token\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:56.280279 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:56.280195    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:56.724072 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:56.724047    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"]
Oct 30 19:00:56.725605 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:00:56.725578    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aac21cb_584a_42ad_9dc6_99aada4cdba1.slice/crio-7ff789c5cf8c6294890f7fc4d505c31f2bb1a520945aaf26cdc20da738513375 WatchSource:0}: Error finding container 7ff789c5cf8c6294890f7fc4d505c31f2bb1a520945aaf26cdc20da738513375: Status 404 returned error can't find the container with id 7ff789c5cf8c6294890f7fc4d505c31f2bb1a520945aaf26cdc20da738513375
Oct 30 19:00:56.728465 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:56.728430    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:00:56.728565 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:56.728504    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:00:56.728565 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:56.728550    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:00:57.640976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:57.635562    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6" event={"ID":"1aac21cb-584a-42ad-9dc6-99aada4cdba1","Type":"ContainerStarted","Data":"d751a058232056c53b8098256b80e74d9a55701ec0380e5b40332de1d1a78149"}
Oct 30 19:00:57.640976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:57.635670    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6" event={"ID":"1aac21cb-584a-42ad-9dc6-99aada4cdba1","Type":"ContainerStarted","Data":"7ff789c5cf8c6294890f7fc4d505c31f2bb1a520945aaf26cdc20da738513375"}
Oct 30 19:00:58.280656 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:58.280615    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:00:58.282004 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:58.281979    5820 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:00:58.282127 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:58.282038    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6" podUID="1aac21cb-584a-42ad-9dc6-99aada4cdba1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:00:59.280746 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:59.280711    5820 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:00:59.281190 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:00:59.280762    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6" podUID="1aac21cb-584a-42ad-9dc6-99aada4cdba1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:01:00.281012 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:00.280940    5820 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:01:00.281508 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:00.281032    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6" podUID="1aac21cb-584a-42ad-9dc6-99aada4cdba1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:01:00.676008 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:00.672637    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:01.281333 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:01.281298    5820 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:01:01.281738 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:01.281363    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6" podUID="1aac21cb-584a-42ad-9dc6-99aada4cdba1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:01:02.280724 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:02.280647    5820 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:01:02.280724 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:02.280701    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6" podUID="1aac21cb-584a-42ad-9dc6-99aada4cdba1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:01:03.298781 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:03.298748    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:01:03.299829 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:03.299801    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:01:03.300614 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:03.300596    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:01:03.523590 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:03.523540    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6" podStartSLOduration=8.523525521 podStartE2EDuration="8.523525521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:57.860514642 +0000 UTC m=+1578.572263340" watchObservedRunningTime="2026-10-30 19:01:03.523525521 +0000 UTC m=+1584.235274218"
Oct 30 19:01:05.885206 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:05.885110    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:06.073298 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:06.073258    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:08.203827 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:01:08.203791    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:01:09.130572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:09.130536    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:09.324015 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:09.323977    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:10.777563 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.777532    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"]
Oct 30 19:01:10.782912 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.782890    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.827122 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.827093    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"]
Oct 30 19:01:10.831327 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.831311    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:10.846051 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.846030    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-workload-certs\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.846164 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.846059    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4ccd454-b219-4d0c-af05-289d90cfe455-istiod-ca-cert\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.846164 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.846093    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-data\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.846164 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.846142    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-podinfo\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.846271 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.846237    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlp2t\" (UniqueName: \"kubernetes.io/projected/e4ccd454-b219-4d0c-af05-289d90cfe455-kube-api-access-hlp2t\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.846311 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.846274    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-token\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.846377 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.846364    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-workload-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.846429 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.846418    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-credential-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.846495 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.846483    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-envoy\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.883251 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.883215    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"]
Oct 30 19:01:10.934481 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.934446    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"]
Oct 30 19:01:10.947629 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.947598    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlp2t\" (UniqueName: \"kubernetes.io/projected/e4ccd454-b219-4d0c-af05-289d90cfe455-kube-api-access-hlp2t\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.947833 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.947651    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-istio-envoy\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:10.947833 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.947681    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-token\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.948041 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.947833    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-workload-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.948041 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.947908    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-workload-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:10.948041 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.947941    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ldhq\" (UniqueName: \"kubernetes.io/projected/370e902d-498f-454d-bacc-1edaf55bcd76-kube-api-access-9ldhq\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:10.948041 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.947987    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-credential-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.948263 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.948246    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-workload-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.948334 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.948300    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-istio-data\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:10.948401 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.948348    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-credential-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.948467 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.948449    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/370e902d-498f-454d-bacc-1edaf55bcd76-istiod-ca-cert\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:10.948522 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.948490    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-envoy\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.948565 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.948538    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-credential-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:10.948621 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.948593    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/370e902d-498f-454d-bacc-1edaf55bcd76-istio-token\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:10.948677 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.948642    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-workload-certs\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:10.948746 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.948732    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/370e902d-498f-454d-bacc-1edaf55bcd76-istio-podinfo\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:10.948805 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.948765    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-workload-certs\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.948862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.948797    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4ccd454-b219-4d0c-af05-289d90cfe455-istiod-ca-cert\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.948916 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.948867    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-data\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.949024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.948920    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-podinfo\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.949128 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.949110    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-workload-certs\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.949365 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.949347    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-data\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.949581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.949561    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4ccd454-b219-4d0c-af05-289d90cfe455-istiod-ca-cert\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.951708 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.951678    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-envoy\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.951843 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.951823    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-podinfo\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:10.971483 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:10.971395    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:11.052797 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.051569    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-token\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:11.053572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.053161    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-workload-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:11.053572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.053420    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ldhq\" (UniqueName: \"kubernetes.io/projected/370e902d-498f-454d-bacc-1edaf55bcd76-kube-api-access-9ldhq\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:11.053572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.053465    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-istio-data\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:11.053572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.057114    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlp2t\" (UniqueName: \"kubernetes.io/projected/e4ccd454-b219-4d0c-af05-289d90cfe455-kube-api-access-hlp2t\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:11.053572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.058388    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-workload-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:11.053572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.060148    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-istio-data\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:11.062358 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.061993    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/370e902d-498f-454d-bacc-1edaf55bcd76-istiod-ca-cert\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:11.062533 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.062512    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/370e902d-498f-454d-bacc-1edaf55bcd76-istiod-ca-cert\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:11.062615 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.062582    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-credential-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:11.062677 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.062668    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/370e902d-498f-454d-bacc-1edaf55bcd76-istio-token\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:11.062730 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.062695    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-workload-certs\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:11.062780 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.062762    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/370e902d-498f-454d-bacc-1edaf55bcd76-istio-podinfo\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:11.062886 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.062872    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-istio-envoy\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:11.064871 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.064846    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-credential-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:11.065109 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.065089    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-workload-certs\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:11.068134 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.068114    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-istio-envoy\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:11.068242 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.068178    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/370e902d-498f-454d-bacc-1edaf55bcd76-istio-podinfo\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:11.099182 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.099154    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:11.119404 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.119378    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ldhq\" (UniqueName: \"kubernetes.io/projected/370e902d-498f-454d-bacc-1edaf55bcd76-kube-api-access-9ldhq\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:11.125138 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.125112    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/370e902d-498f-454d-bacc-1edaf55bcd76-istio-token\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:11.145928 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.145896    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:11.537229 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.537153    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"]
Oct 30 19:01:11.570608 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:01:11.570580    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4ccd454_b219_4d0c_af05_289d90cfe455.slice/crio-f1c7cefd357ceeb3b3162f275b424d02fa82b5eb5b30e696440d0ca93c550da8 WatchSource:0}: Error finding container f1c7cefd357ceeb3b3162f275b424d02fa82b5eb5b30e696440d0ca93c550da8: Status 404 returned error can't find the container with id f1c7cefd357ceeb3b3162f275b424d02fa82b5eb5b30e696440d0ca93c550da8
Oct 30 19:01:11.573070 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.573036    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:01:11.573151 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.573108    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:01:11.573151 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.573136    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:01:11.588251 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.588231    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"]
Oct 30 19:01:11.590064 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:01:11.590035    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod370e902d_498f_454d_bacc_1edaf55bcd76.slice/crio-02836f181faa6236826d6e029ead6e6f4ab02735488136f2cce1a31fb0c0ffec WatchSource:0}: Error finding container 02836f181faa6236826d6e029ead6e6f4ab02735488136f2cce1a31fb0c0ffec: Status 404 returned error can't find the container with id 02836f181faa6236826d6e029ead6e6f4ab02735488136f2cce1a31fb0c0ffec
Oct 30 19:01:11.592689 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.592640    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:01:11.592736 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.592694    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:01:11.592736 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.592724    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:01:11.799132 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.799094    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b" event={"ID":"370e902d-498f-454d-bacc-1edaf55bcd76","Type":"ContainerStarted","Data":"83d1b426c700e2a25af84cd3101733b8de805c951f295819d6a831f9e7d663c8"}
Oct 30 19:01:11.799572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.799156    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b" event={"ID":"370e902d-498f-454d-bacc-1edaf55bcd76","Type":"ContainerStarted","Data":"02836f181faa6236826d6e029ead6e6f4ab02735488136f2cce1a31fb0c0ffec"}
Oct 30 19:01:11.801192 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.801155    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg" event={"ID":"e4ccd454-b219-4d0c-af05-289d90cfe455","Type":"ContainerStarted","Data":"694e1f97c38f20709977d2ba4b28ecb91eee74347f4cc0cbf2c77f8261c5cb07"}
Oct 30 19:01:11.801319 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.801202    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg" event={"ID":"e4ccd454-b219-4d0c-af05-289d90cfe455","Type":"ContainerStarted","Data":"f1c7cefd357ceeb3b3162f275b424d02fa82b5eb5b30e696440d0ca93c550da8"}
Oct 30 19:01:11.927891 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:11.927834    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b" podStartSLOduration=1.927818186 podStartE2EDuration="1.927818186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:11.922839242 +0000 UTC m=+1592.634587938" watchObservedRunningTime="2026-10-30 19:01:11.927818186 +0000 UTC m=+1592.639566953"
Oct 30 19:01:12.048980 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:12.048904    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg" podStartSLOduration=2.048888585 podStartE2EDuration="2.048888585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:12.0439933 +0000 UTC m=+1592.755741990" watchObservedRunningTime="2026-10-30 19:01:12.048888585 +0000 UTC m=+1592.760637281"
Oct 30 19:01:12.099517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:12.099393    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:12.101018 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:12.100994    5820 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 19:01:12.101143 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:12.101046    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg" podUID="e4ccd454-b219-4d0c-af05-289d90cfe455" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 19:01:12.150974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:12.149774    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:12.166975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:12.164104    5820 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:01:12.166975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:12.164172    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b" podUID="370e902d-498f-454d-bacc-1edaf55bcd76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 19:01:13.100025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:13.099987    5820 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 19:01:13.100470 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:13.100058    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg" podUID="e4ccd454-b219-4d0c-af05-289d90cfe455" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 19:01:13.161984 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:13.159056    5820 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:01:13.161984 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:13.159118    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b" podUID="370e902d-498f-454d-bacc-1edaf55bcd76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 19:01:14.101573 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:14.101538    5820 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 19:01:14.102274 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:14.102221    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg" podUID="e4ccd454-b219-4d0c-af05-289d90cfe455" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 19:01:14.153975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:14.151325    5820 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:01:14.153975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:14.151405    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b" podUID="370e902d-498f-454d-bacc-1edaf55bcd76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 19:01:15.109568 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:15.107605    5820 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 19:01:15.109568 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:15.107671    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg" podUID="e4ccd454-b219-4d0c-af05-289d90cfe455" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 19:01:15.147500 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:15.147358    5820 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:01:15.147500 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:15.147437    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b" podUID="370e902d-498f-454d-bacc-1edaf55bcd76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 19:01:15.300991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:15.300933    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"]
Oct 30 19:01:15.301771 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:15.301742    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4" podUID="0d3dbc80-0283-496b-a153-8d7ae01b7946" containerName="maas-api" containerID="cri-o://61b2c8d8e0ea6f551a779bf4e9582e6d85f6f4b65a097ddb787c52f3fe1d89e6" gracePeriod=30
Oct 30 19:01:15.458061 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:15.449832    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:16.011158 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:16.011019    5820 generic.go:356] "Generic (PLEG): container finished" podID="0d3dbc80-0283-496b-a153-8d7ae01b7946" containerID="61b2c8d8e0ea6f551a779bf4e9582e6d85f6f4b65a097ddb787c52f3fe1d89e6" exitCode=0
Oct 30 19:01:16.011158 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:16.011091    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4" event={"ID":"0d3dbc80-0283-496b-a153-8d7ae01b7946","Type":"ContainerDied","Data":"61b2c8d8e0ea6f551a779bf4e9582e6d85f6f4b65a097ddb787c52f3fe1d89e6"}
Oct 30 19:01:16.105977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:16.104732    5820 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 19:01:16.105977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:16.104798    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg" podUID="e4ccd454-b219-4d0c-af05-289d90cfe455" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 19:01:16.120075 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:16.115138    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:16.147384 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:16.147346    5820 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:01:16.147676 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:16.147646    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b" podUID="370e902d-498f-454d-bacc-1edaf55bcd76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 19:01:16.827030 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:16.826998    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"
Oct 30 19:01:16.922413 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:16.922375    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0d3dbc80-0283-496b-a153-8d7ae01b7946-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3dbc80-0283-496b-a153-8d7ae01b7946-openshift-service-ca\") pod \"0d3dbc80-0283-496b-a153-8d7ae01b7946\" (UID: \"0d3dbc80-0283-496b-a153-8d7ae01b7946\") "
Oct 30 19:01:16.922592 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:16.922487    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0d3dbc80-0283-496b-a153-8d7ae01b7946-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d3dbc80-0283-496b-a153-8d7ae01b7946-maas-api-tls\") pod \"0d3dbc80-0283-496b-a153-8d7ae01b7946\" (UID: \"0d3dbc80-0283-496b-a153-8d7ae01b7946\") "
Oct 30 19:01:16.922636 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:16.922595    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0d3dbc80-0283-496b-a153-8d7ae01b7946-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d3dbc80-0283-496b-a153-8d7ae01b7946-metrics-certs\") pod \"0d3dbc80-0283-496b-a153-8d7ae01b7946\" (UID: \"0d3dbc80-0283-496b-a153-8d7ae01b7946\") "
Oct 30 19:01:16.922671 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:16.922650    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d3dbc80-0283-496b-a153-8d7ae01b7946-kube-api-access-d5drd\" (UniqueName: \"kubernetes.io/projected/0d3dbc80-0283-496b-a153-8d7ae01b7946-kube-api-access-d5drd\") pod \"0d3dbc80-0283-496b-a153-8d7ae01b7946\" (UID: \"0d3dbc80-0283-496b-a153-8d7ae01b7946\") "
Oct 30 19:01:16.923641 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:16.923618    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d3dbc80-0283-496b-a153-8d7ae01b7946-openshift-service-ca" pod "0d3dbc80-0283-496b-a153-8d7ae01b7946" (UID: "0d3dbc80-0283-496b-a153-8d7ae01b7946"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:01:16.925314 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:16.925289    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d3dbc80-0283-496b-a153-8d7ae01b7946-kube-api-access-d5drd" pod "0d3dbc80-0283-496b-a153-8d7ae01b7946" (UID: "0d3dbc80-0283-496b-a153-8d7ae01b7946"). InnerVolumeSpecName "kube-api-access-d5drd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:16.925996 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:16.925976    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d3dbc80-0283-496b-a153-8d7ae01b7946-metrics-certs" pod "0d3dbc80-0283-496b-a153-8d7ae01b7946" (UID: "0d3dbc80-0283-496b-a153-8d7ae01b7946"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:01:16.926044 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:16.925985    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d3dbc80-0283-496b-a153-8d7ae01b7946-maas-api-tls" pod "0d3dbc80-0283-496b-a153-8d7ae01b7946" (UID: "0d3dbc80-0283-496b-a153-8d7ae01b7946"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:01:17.029884 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:17.029796    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d3dbc80-0283-496b-a153-8d7ae01b7946-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:01:17.030265 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:17.030241    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5drd\" (UniqueName: \"kubernetes.io/projected/0d3dbc80-0283-496b-a153-8d7ae01b7946-kube-api-access-d5drd\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:01:17.030424 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:17.030407    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3dbc80-0283-496b-a153-8d7ae01b7946-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:01:17.030532 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:17.030518    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d3dbc80-0283-496b-a153-8d7ae01b7946-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:01:17.131050 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:17.129832    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4" event={"ID":"0d3dbc80-0283-496b-a153-8d7ae01b7946","Type":"ContainerDied","Data":"da9ec89cc8b82de3a055ac5d8b0fdfe7117b29dfc655007de9d48cb4d52040b6"}
Oct 30 19:01:17.131050 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:17.129995    5820 scope.go:122] "RemoveContainer" containerID="61b2c8d8e0ea6f551a779bf4e9582e6d85f6f4b65a097ddb787c52f3fe1d89e6"
Oct 30 19:01:17.131050 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:17.130290    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"
Oct 30 19:01:17.131693 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:17.131525    5820 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 19:01:17.131693 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:17.131572    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg" podUID="e4ccd454-b219-4d0c-af05-289d90cfe455" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 19:01:17.176184 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:17.170854    5820 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:01:17.176184 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:17.170929    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b" podUID="370e902d-498f-454d-bacc-1edaf55bcd76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 19:01:17.177190 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:17.177055    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:17.413975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:17.413319    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"]
Oct 30 19:01:17.424883 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:17.424843    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-10236a-6bd8dc998b-cg5k4"]
Oct 30 19:01:17.478648 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:17.478607    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:17.513976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:17.507357    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d3dbc80-0283-496b-a153-8d7ae01b7946" path="/var/lib/kubelet/pods/0d3dbc80-0283-496b-a153-8d7ae01b7946/volumes"
Oct 30 19:01:17.706041 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:17.705932    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:18.104359 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:18.104323    5820 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 19:01:18.104667 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:18.104639    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg" podUID="e4ccd454-b219-4d0c-af05-289d90cfe455" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 19:01:18.153985 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:18.153282    5820 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:01:18.153985 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:18.153341    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b" podUID="370e902d-498f-454d-bacc-1edaf55bcd76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 19:01:18.202977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:18.201663    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:18.618274 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:18.618233    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:19.104809 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:19.104782    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:19.105425 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:19.105409    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:19.106074 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:19.106056    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:01:19.163348 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:19.163318    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:19.163800 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:19.163610    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:19.164311 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:19.164296    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:01:22.770425 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:22.770299    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:24.248695 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.248665    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"]
Oct 30 19:01:24.249370 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.249356    5820 cpu_manager.go:414] "containerMap: removing container" podUID="0d3dbc80-0283-496b-a153-8d7ae01b7946" containerName="maas-api"
Oct 30 19:01:24.249426 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.249372    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d3dbc80-0283-496b-a153-8d7ae01b7946" containerName="maas-api"
Oct 30 19:01:24.249501 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.249492    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d3dbc80-0283-496b-a153-8d7ae01b7946" containerName="maas-api"
Oct 30 19:01:24.251857 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.251837    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.259811 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.259764    5820 generic.go:356] "Generic (PLEG): container finished" podID="5dece09c-bff0-403f-a991-b204dc1f0f4c" containerID="44d724d76361f69e6e6b274c0cd909be0f1d819a288bc7adc57196990cfce6f9" exitCode=0
Oct 30 19:01:24.259888 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.259821    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms" event={"ID":"5dece09c-bff0-403f-a991-b204dc1f0f4c","Type":"ContainerDied","Data":"44d724d76361f69e6e6b274c0cd909be0f1d819a288bc7adc57196990cfce6f9"}
Oct 30 19:01:24.260271 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.260254    5820 scope.go:122] "RemoveContainer" containerID="44d724d76361f69e6e6b274c0cd909be0f1d819a288bc7adc57196990cfce6f9"
Oct 30 19:01:24.261779 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.261748    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:01:24.261877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.261822    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:01:24.261877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.261872    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:01:24.303852 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.303821    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-token\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.304004 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.303885    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-data\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.304060 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.304028    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istiod-ca-cert\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.304117 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.304098    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-podinfo\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.304239 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.304218    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcnmb\" (UniqueName: \"kubernetes.io/projected/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-kube-api-access-wcnmb\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.304309 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.304272    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-workload-certs\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.304386 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.304324    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-credential-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.304455 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.304432    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-envoy\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.304542 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.304525    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-workload-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.368207 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.367242    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"]
Oct 30 19:01:24.416011 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.415885    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istiod-ca-cert\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.416011 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.416008    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-podinfo\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.416512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.416046    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcnmb\" (UniqueName: \"kubernetes.io/projected/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-kube-api-access-wcnmb\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.416512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.416103    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-workload-certs\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.416512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.416131    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-credential-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.416512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.416185    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-envoy\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.416512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.416272    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-workload-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.416512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.416371    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-token\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.416512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.416463    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-data\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.416987 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.416926    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-data\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.417609 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.417580    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istiod-ca-cert\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.417719 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.417630    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-credential-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.418474 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.418425    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-workload-certs\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.418800 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.418774    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-workload-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.422695 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.422669    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-podinfo\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.426193 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.426149    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-envoy\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.481014 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.480982    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcnmb\" (UniqueName: \"kubernetes.io/projected/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-kube-api-access-wcnmb\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.481163 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.480984    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-token\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:24.566201 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:24.566174    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:25.030970 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:25.027376    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"]
Oct 30 19:01:25.030970 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:01:25.028530    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d11b7b9_1021_4bd9_8f74_ea0e091a2a82.slice/crio-130c792cbef1aabdb48d69be1ba013ada3c743245284a32c77f74f1da7fc0edc WatchSource:0}: Error finding container 130c792cbef1aabdb48d69be1ba013ada3c743245284a32c77f74f1da7fc0edc: Status 404 returned error can't find the container with id 130c792cbef1aabdb48d69be1ba013ada3c743245284a32c77f74f1da7fc0edc
Oct 30 19:01:25.031616 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:25.031576    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:01:25.031725 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:25.031660    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:01:25.031725 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:25.031699    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:01:25.269141 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:25.269092    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs" event={"ID":"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82","Type":"ContainerStarted","Data":"343855aae3287a8e82cc8c3735eac6ad8af7e37d01a80395f2a6192735048f8d"}
Oct 30 19:01:25.269560 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:25.269195    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs" event={"ID":"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82","Type":"ContainerStarted","Data":"130c792cbef1aabdb48d69be1ba013ada3c743245284a32c77f74f1da7fc0edc"}
Oct 30 19:01:25.271799 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:25.271774    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms" event={"ID":"5dece09c-bff0-403f-a991-b204dc1f0f4c","Type":"ContainerStarted","Data":"1dc2bec8cabd6e2680df9b1827d1820604a1ac66ea0538d17ced37930401dd37"}
Oct 30 19:01:25.396399 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:25.396336    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs" podStartSLOduration=1.396321963 podStartE2EDuration="1.396321963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:25.392393183 +0000 UTC m=+1606.104141876" watchObservedRunningTime="2026-10-30 19:01:25.396321963 +0000 UTC m=+1606.108070660"
Oct 30 19:01:25.567210 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:25.567161    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:25.627090 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:25.627047    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:25.700399 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:25.700310    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 19:01:25.702651 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:25.702617    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:01:25.702927 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:25.702894    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms" podUID="5dece09c-bff0-403f-a991-b204dc1f0f4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 19:01:25.723719 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:25.723686    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 19:01:25.812619 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:25.812593    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:26.020037 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:26.019204    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"]
Oct 30 19:01:26.020037 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:26.019480    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp" podUID="d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" containerName="istio-proxy" containerID="cri-o://fe09d43c20db9515a706adcc41f5f15e9b5f9de9fdbf16b99b5649743fff832b" gracePeriod=30
Oct 30 19:01:26.024858 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:26.023912    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:26.586973 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:26.586105    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 19:01:26.586973 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:26.586168    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs" podUID="9d11b7b9-1021-4bd9-8f74-ea0e091a2a82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 19:01:26.697275 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:26.697242    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:01:26.697447 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:26.697319    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms" podUID="5dece09c-bff0-403f-a991-b204dc1f0f4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 19:01:27.578806 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:27.578366    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 19:01:27.578806 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:27.578427    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs" podUID="9d11b7b9-1021-4bd9-8f74-ea0e091a2a82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 19:01:27.698056 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:27.698018    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:01:27.698459 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:27.698072    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms" podUID="5dece09c-bff0-403f-a991-b204dc1f0f4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 19:01:28.231979 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:28.225920    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:28.572599 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:28.572565    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 19:01:28.572869 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:28.572842    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs" podUID="9d11b7b9-1021-4bd9-8f74-ea0e091a2a82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 19:01:28.697911 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:28.697876    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:01:28.698108 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:28.697928    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms" podUID="5dece09c-bff0-403f-a991-b204dc1f0f4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 19:01:29.572977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:29.572219    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 19:01:29.572977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:29.572280    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs" podUID="9d11b7b9-1021-4bd9-8f74-ea0e091a2a82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 19:01:29.697174 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:29.697139    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:01:29.697348 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:29.697210    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms" podUID="5dece09c-bff0-403f-a991-b204dc1f0f4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 19:01:29.810243 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:29.810215    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:30.585970 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:30.581492    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 19:01:30.585970 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:30.581553    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs" podUID="9d11b7b9-1021-4bd9-8f74-ea0e091a2a82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 19:01:30.698027 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:30.697992    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:01:30.698326 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:30.698296    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms" podUID="5dece09c-bff0-403f-a991-b204dc1f0f4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 19:01:30.979029 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:30.973255    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:31.200444 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.200205    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:31.355331 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.355305    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 19:01:31.406343 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.406309    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-workload-certs\") pod \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") "
Oct 30 19:01:31.406547 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.406380    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-workload-socket\") pod \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") "
Oct 30 19:01:31.406547 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.406449    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-data\") pod \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") "
Oct 30 19:01:31.406547 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.406496    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-credential-socket\") pod \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") "
Oct 30 19:01:31.406547 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.406528    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-podinfo\") pod \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") "
Oct 30 19:01:31.407324 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.406569    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istiod-ca-cert\") pod \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") "
Oct 30 19:01:31.407324 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.406628    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-workload-certs" pod "d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" (UID: "d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:31.407324 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.406674    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-kube-api-access-r9z8m\" (UniqueName: \"kubernetes.io/projected/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-kube-api-access-r9z8m\") pod \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") "
Oct 30 19:01:31.407324 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.406803    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-envoy\") pod \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") "
Oct 30 19:01:31.407324 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.406859    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-data" pod "d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" (UID: "d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:31.407324 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.406891    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-token\" (UniqueName: \"kubernetes.io/projected/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-token\") pod \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\" (UID: \"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97\") "
Oct 30 19:01:31.407324 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.406992    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istiod-ca-cert" pod "d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" (UID: "d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:01:31.407324 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.407025    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-credential-socket" pod "d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" (UID: "d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:31.407324 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.407201    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-workload-socket" pod "d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" (UID: "d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:31.408018 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.407801    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:01:31.408018 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.407826    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:01:31.408018 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.407841    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:01:31.408018 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.407855    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:01:31.408018 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.407869    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:01:31.409192 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.409164    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-kube-api-access-r9z8m" pod "d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" (UID: "d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97"). InnerVolumeSpecName "kube-api-access-r9z8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:31.409371 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.409348    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-token" pod "d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" (UID: "d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:31.409505 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.409488    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-podinfo" pod "d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" (UID: "d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:01:31.409941 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.409918    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-envoy" pod "d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" (UID: "d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:31.440690 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.440656    5820 generic.go:356] "Generic (PLEG): container finished" podID="d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" containerID="fe09d43c20db9515a706adcc41f5f15e9b5f9de9fdbf16b99b5649743fff832b" exitCode=0
Oct 30 19:01:31.440827 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.440765    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp" event={"ID":"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97","Type":"ContainerDied","Data":"fe09d43c20db9515a706adcc41f5f15e9b5f9de9fdbf16b99b5649743fff832b"}
Oct 30 19:01:31.440827 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.440813    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp" event={"ID":"d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97","Type":"ContainerDied","Data":"ddaa1187cfba65ad5ce12cf3150955372765ddc8421564d9209f66eb30c11219"}
Oct 30 19:01:31.440900 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.440845    5820 scope.go:122] "RemoveContainer" containerID="fe09d43c20db9515a706adcc41f5f15e9b5f9de9fdbf16b99b5649743fff832b"
Oct 30 19:01:31.441072 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.441056    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"
Oct 30 19:01:31.454484 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.454461    5820 scope.go:122] "RemoveContainer" containerID="cb3a336f58660453f2b18cbcb5d6a31347bf0ac0d0bb845060361484785de024"
Oct 30 19:01:31.463894 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.463876    5820 scope.go:122] "RemoveContainer" containerID="fe09d43c20db9515a706adcc41f5f15e9b5f9de9fdbf16b99b5649743fff832b"
Oct 30 19:01:31.464176 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:01:31.464149    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe09d43c20db9515a706adcc41f5f15e9b5f9de9fdbf16b99b5649743fff832b\": container with ID starting with fe09d43c20db9515a706adcc41f5f15e9b5f9de9fdbf16b99b5649743fff832b not found: ID does not exist" containerID="fe09d43c20db9515a706adcc41f5f15e9b5f9de9fdbf16b99b5649743fff832b"
Oct 30 19:01:31.464281 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.464186    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe09d43c20db9515a706adcc41f5f15e9b5f9de9fdbf16b99b5649743fff832b"} err="failed to get container status \"fe09d43c20db9515a706adcc41f5f15e9b5f9de9fdbf16b99b5649743fff832b\": rpc error: code = NotFound desc = could not find container \"fe09d43c20db9515a706adcc41f5f15e9b5f9de9fdbf16b99b5649743fff832b\": container with ID starting with fe09d43c20db9515a706adcc41f5f15e9b5f9de9fdbf16b99b5649743fff832b not found: ID does not exist"
Oct 30 19:01:31.464281 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.464204    5820 scope.go:122] "RemoveContainer" containerID="cb3a336f58660453f2b18cbcb5d6a31347bf0ac0d0bb845060361484785de024"
Oct 30 19:01:31.464666 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:01:31.464505    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb3a336f58660453f2b18cbcb5d6a31347bf0ac0d0bb845060361484785de024\": container with ID starting with cb3a336f58660453f2b18cbcb5d6a31347bf0ac0d0bb845060361484785de024 not found: ID does not exist" containerID="cb3a336f58660453f2b18cbcb5d6a31347bf0ac0d0bb845060361484785de024"
Oct 30 19:01:31.464666 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.464537    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb3a336f58660453f2b18cbcb5d6a31347bf0ac0d0bb845060361484785de024"} err="failed to get container status \"cb3a336f58660453f2b18cbcb5d6a31347bf0ac0d0bb845060361484785de024\": rpc error: code = NotFound desc = could not find container \"cb3a336f58660453f2b18cbcb5d6a31347bf0ac0d0bb845060361484785de024\": container with ID starting with cb3a336f58660453f2b18cbcb5d6a31347bf0ac0d0bb845060361484785de024 not found: ID does not exist"
Oct 30 19:01:31.510314 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.510278    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:01:31.510314 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.510316    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:01:31.510595 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.510352    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:01:31.510595 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.510372    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9z8m\" (UniqueName: \"kubernetes.io/projected/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97-kube-api-access-r9z8m\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:01:31.593020 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.592904    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:31.594415 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.594380    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:31.595151 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.595131    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:01:31.600399 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.600342    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"]
Oct 30 19:01:31.605236 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.605201    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-10236a-openshift-default-67dbc7c8f9-b74qp"]
Oct 30 19:01:31.701660 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.701630    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 19:01:31.702635 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:31.702613    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 19:01:33.455555 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:33.455508    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:33.470842 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:33.470798    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" path="/var/lib/kubelet/pods/d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97/volumes"
Oct 30 19:01:34.795894 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:34.795845    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:35.032989 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:35.032914    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:38.182022 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:38.180965    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:38.205698 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:01:38.205660    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:01:38.632133 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:38.632091    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:38.736971 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:38.721220    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn"]
Oct 30 19:01:38.738291 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:38.738259    5820 cpu_manager.go:414] "containerMap: removing container" podUID="d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" containerName="istio-proxy"
Oct 30 19:01:38.738455 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:38.738443    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" containerName="istio-proxy"
Oct 30 19:01:38.738785 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:38.738769    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" containerName="istio-proxy"
Oct 30 19:01:38.776893 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:38.776860    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn"
Oct 30 19:01:38.808983 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:38.791263    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn"]
Oct 30 19:01:38.897915 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:38.895156    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qd68\" (UniqueName: \"kubernetes.io/projected/efd8ffd1-02a7-4096-8206-de01acd83987-kube-api-access-9qd68\") pod \"payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn\" (UID: \"efd8ffd1-02a7-4096-8206-de01acd83987\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn"
Oct 30 19:01:38.897915 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:38.895250    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efd8ffd1-02a7-4096-8206-de01acd83987-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn\" (UID: \"efd8ffd1-02a7-4096-8206-de01acd83987\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn"
Oct 30 19:01:38.980011 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:38.951369    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-82305a-dockercfg-vt92q\""
Oct 30 19:01:38.980011 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:38.951492    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-82305a\""
Oct 30 19:01:39.006070 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:38.996840    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qd68\" (UniqueName: \"kubernetes.io/projected/efd8ffd1-02a7-4096-8206-de01acd83987-kube-api-access-9qd68\") pod \"payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn\" (UID: \"efd8ffd1-02a7-4096-8206-de01acd83987\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn"
Oct 30 19:01:39.006070 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:38.996929    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efd8ffd1-02a7-4096-8206-de01acd83987-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn\" (UID: \"efd8ffd1-02a7-4096-8206-de01acd83987\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn"
Oct 30 19:01:39.016100 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:39.013201    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efd8ffd1-02a7-4096-8206-de01acd83987-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn\" (UID: \"efd8ffd1-02a7-4096-8206-de01acd83987\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn"
Oct 30 19:01:39.172621 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:39.172193    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr"]
Oct 30 19:01:39.173346 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:39.173322    5820 cpu_manager.go:414] "containerMap: removing container" podUID="d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" containerName="istio-proxy"
Oct 30 19:01:39.173346 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:39.173346    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" containerName="istio-proxy"
Oct 30 19:01:39.173548 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:39.173537    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0ab86cb-79ad-43d5-ae1b-13b4e0bd1c97" containerName="istio-proxy"
Oct 30 19:01:39.199077 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:39.199041    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:39.229005 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:39.228976    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qd68\" (UniqueName: \"kubernetes.io/projected/efd8ffd1-02a7-4096-8206-de01acd83987-kube-api-access-9qd68\") pod \"payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn\" (UID: \"efd8ffd1-02a7-4096-8206-de01acd83987\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn"
Oct 30 19:01:39.229576 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:39.229555    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr"
Oct 30 19:01:39.257424 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:39.257377    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn"
Oct 30 19:01:39.301631 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:39.299292    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr"]
Oct 30 19:01:39.301631 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:39.300598    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hfrw\" (UniqueName: \"kubernetes.io/projected/6a9f6116-2dd0-4f64-b212-0acf561d5307-kube-api-access-4hfrw\") pod \"payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr\" (UID: \"6a9f6116-2dd0-4f64-b212-0acf561d5307\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr"
Oct 30 19:01:39.301631 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:39.300729    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a9f6116-2dd0-4f64-b212-0acf561d5307-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr\" (UID: \"6a9f6116-2dd0-4f64-b212-0acf561d5307\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr"
Oct 30 19:01:39.401870 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:39.401840    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hfrw\" (UniqueName: \"kubernetes.io/projected/6a9f6116-2dd0-4f64-b212-0acf561d5307-kube-api-access-4hfrw\") pod \"payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr\" (UID: \"6a9f6116-2dd0-4f64-b212-0acf561d5307\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr"
Oct 30 19:01:39.402050 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:39.401928    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a9f6116-2dd0-4f64-b212-0acf561d5307-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr\" (UID: \"6a9f6116-2dd0-4f64-b212-0acf561d5307\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr"
Oct 30 19:01:39.402611 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:39.402584    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a9f6116-2dd0-4f64-b212-0acf561d5307-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr\" (UID: \"6a9f6116-2dd0-4f64-b212-0acf561d5307\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr"
Oct 30 19:01:39.464077 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:39.463991    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hfrw\" (UniqueName: \"kubernetes.io/projected/6a9f6116-2dd0-4f64-b212-0acf561d5307-kube-api-access-4hfrw\") pod \"payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr\" (UID: \"6a9f6116-2dd0-4f64-b212-0acf561d5307\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr"
Oct 30 19:01:39.586798 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:39.586767    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr"
Oct 30 19:01:39.694204 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:39.694161    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn"]
Oct 30 19:01:39.695252 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:01:39.695221    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefd8ffd1_02a7_4096_8206_de01acd83987.slice/crio-ebee7b165d8a34338e886088328b0f212d9035c9f91a331fa3d288eb10b652e9 WatchSource:0}: Error finding container ebee7b165d8a34338e886088328b0f212d9035c9f91a331fa3d288eb10b652e9: Status 404 returned error can't find the container with id ebee7b165d8a34338e886088328b0f212d9035c9f91a331fa3d288eb10b652e9
Oct 30 19:01:39.969111 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:39.969065    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr"]
Oct 30 19:01:39.970327 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:01:39.970301    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a9f6116_2dd0_4f64_b212_0acf561d5307.slice/crio-d88eccf037741dd422ad40d1268877481865b7d1707015fe751834781fd4089a WatchSource:0}: Error finding container d88eccf037741dd422ad40d1268877481865b7d1707015fe751834781fd4089a: Status 404 returned error can't find the container with id d88eccf037741dd422ad40d1268877481865b7d1707015fe751834781fd4089a
Oct 30 19:01:40.685118 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:40.685080    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr" event={"ID":"6a9f6116-2dd0-4f64-b212-0acf561d5307","Type":"ContainerStarted","Data":"58c3898193a07c6134acfb024d8f0669218b3c962b3a2a3eba42b86450a31401"}
Oct 30 19:01:40.685612 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:40.685135    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr" event={"ID":"6a9f6116-2dd0-4f64-b212-0acf561d5307","Type":"ContainerStarted","Data":"d88eccf037741dd422ad40d1268877481865b7d1707015fe751834781fd4089a"}
Oct 30 19:01:40.685612 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:40.685269    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr"
Oct 30 19:01:40.686834 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:40.686809    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn" event={"ID":"efd8ffd1-02a7-4096-8206-de01acd83987","Type":"ContainerStarted","Data":"ff05bf57900c1530ac2401e9a92088d51e690bcda11ec35f6006af60d95cc5ce"}
Oct 30 19:01:40.686969 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:40.686849    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn" event={"ID":"efd8ffd1-02a7-4096-8206-de01acd83987","Type":"ContainerStarted","Data":"ebee7b165d8a34338e886088328b0f212d9035c9f91a331fa3d288eb10b652e9"}
Oct 30 19:01:40.686969 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:40.686890    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn"
Oct 30 19:01:40.804146 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:40.804088    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr" podStartSLOduration=2.804073588 podStartE2EDuration="2.804073588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:40.80244144 +0000 UTC m=+1621.514190138" watchObservedRunningTime="2026-10-30 19:01:40.804073588 +0000 UTC m=+1621.515822284"
Oct 30 19:01:44.431825 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:44.431791    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:44.685817 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:44.685712    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn" podStartSLOduration=6.685692017 podStartE2EDuration="6.685692017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:40.949814054 +0000 UTC m=+1621.661562752" watchObservedRunningTime="2026-10-30 19:01:44.685692017 +0000 UTC m=+1625.397440715"
Oct 30 19:01:44.688628 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:44.688605    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7112-8c3e32"]
Oct 30 19:01:44.693031 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:44.693012    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7112-8c3e32"
Oct 30 19:01:44.714528 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:44.714496    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sghzs\" (UniqueName: \"kubernetes.io/projected/6329d3ce-2871-4e1f-b557-399ccadd7d0f-kube-api-access-sghzs\") pod \"test-curl-7112-8c3e32\" (UID: \"6329d3ce-2871-4e1f-b557-399ccadd7d0f\") " pod="opendatahub/test-curl-7112-8c3e32"
Oct 30 19:01:44.748349 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:44.748321    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7q8sq\""
Oct 30 19:01:44.756503 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:44.756474    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7112-8c3e32"]
Oct 30 19:01:44.815605 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:44.815554    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sghzs\" (UniqueName: \"kubernetes.io/projected/6329d3ce-2871-4e1f-b557-399ccadd7d0f-kube-api-access-sghzs\") pod \"test-curl-7112-8c3e32\" (UID: \"6329d3ce-2871-4e1f-b557-399ccadd7d0f\") " pod="opendatahub/test-curl-7112-8c3e32"
Oct 30 19:01:44.876500 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:44.876469    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sghzs\" (UniqueName: \"kubernetes.io/projected/6329d3ce-2871-4e1f-b557-399ccadd7d0f-kube-api-access-sghzs\") pod \"test-curl-7112-8c3e32\" (UID: \"6329d3ce-2871-4e1f-b557-399ccadd7d0f\") " pod="opendatahub/test-curl-7112-8c3e32"
Oct 30 19:01:45.006717 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:45.006629    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7112-8c3e32"
Oct 30 19:01:45.386789 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:45.386759    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7112-8c3e32"]
Oct 30 19:01:45.387102 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:01:45.387081    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6329d3ce_2871_4e1f_b557_399ccadd7d0f.slice/crio-851162d49788c1f80828988dd0026795861e0be789eb49e7de5b3f6920872b45 WatchSource:0}: Error finding container 851162d49788c1f80828988dd0026795861e0be789eb49e7de5b3f6920872b45: Status 404 returned error can't find the container with id 851162d49788c1f80828988dd0026795861e0be789eb49e7de5b3f6920872b45
Oct 30 19:01:45.626999 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:45.623771    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:45.730618 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:45.730519    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7112-8c3e32" event={"ID":"6329d3ce-2871-4e1f-b557-399ccadd7d0f","Type":"ContainerStarted","Data":"851162d49788c1f80828988dd0026795861e0be789eb49e7de5b3f6920872b45"}
Oct 30 19:01:48.106155 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:48.106116    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r"]
Oct 30 19:01:48.511869 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:48.511798    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:48.739495 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:48.739438    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:48.753545 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:48.753508    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7112-8c3e32" event={"ID":"6329d3ce-2871-4e1f-b557-399ccadd7d0f","Type":"ContainerStarted","Data":"69e9ccfaa3b0b6a707b45b04c509b079de4e8ef812baad4e2d439e7f73a255f9"}
Oct 30 19:01:48.867566 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:48.867504    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7112-8c3e32" podStartSLOduration=2.5431186869999998 podStartE2EDuration="4.867487622s" totalImagesPullingTime="2.324368935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:48.866376691 +0000 UTC m=+1629.578125387" watchObservedRunningTime="2026-10-30 19:01:48.867487622 +0000 UTC m=+1629.579236320"
Oct 30 19:01:49.773967 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:49.773597    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:50.620564 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:50.620521    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7112-8c3e32"]
Oct 30 19:01:50.625992 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:50.625966    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7112-8c3e32"]
Oct 30 19:01:51.094894 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:51.094843    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7112-8c3e32" podUID="6329d3ce-2871-4e1f-b557-399ccadd7d0f" containerName="test-curl-7112-8c3e32" containerID="cri-o://69e9ccfaa3b0b6a707b45b04c509b079de4e8ef812baad4e2d439e7f73a255f9" gracePeriod=2
Oct 30 19:01:51.145933 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:01:51.145892    5820 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7112-8c3e32\" is forbidden: User \"system:node:ip-10-0-133-80.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-80.us-west-1.compute.internal' and this object" podUID="6329d3ce-2871-4e1f-b557-399ccadd7d0f" pod="opendatahub/test-curl-7112-8c3e32"
Oct 30 19:01:51.697128 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:51.697098    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn"
Oct 30 19:01:51.697337 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:51.697319    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr"
Oct 30 19:01:53.107086 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:53.107049    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r" podUID="7454f83e-8a9e-4fcf-8468-6cd18c64a8b8" containerName="payload-processing" containerID="cri-o://58addef4eef7e5b1c52c2bd6b8be99dd153528b97eae32cd0ffd3cec10c0b9ba" gracePeriod=25
Oct 30 19:01:53.392347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:53.392319    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r"
Oct 30 19:01:53.395685 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:53.395664    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7112-8c3e32"
Oct 30 19:01:53.481089 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:53.481053    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6329d3ce-2871-4e1f-b557-399ccadd7d0f-kube-api-access-sghzs\" (UniqueName: \"kubernetes.io/projected/6329d3ce-2871-4e1f-b557-399ccadd7d0f-kube-api-access-sghzs\") pod \"6329d3ce-2871-4e1f-b557-399ccadd7d0f\" (UID: \"6329d3ce-2871-4e1f-b557-399ccadd7d0f\") "
Oct 30 19:01:53.481235 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:53.481223    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7454f83e-8a9e-4fcf-8468-6cd18c64a8b8-kube-api-access-hj28b\" (UniqueName: \"kubernetes.io/projected/7454f83e-8a9e-4fcf-8468-6cd18c64a8b8-kube-api-access-hj28b\") pod \"7454f83e-8a9e-4fcf-8468-6cd18c64a8b8\" (UID: \"7454f83e-8a9e-4fcf-8468-6cd18c64a8b8\") "
Oct 30 19:01:53.481292 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:53.481253    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7454f83e-8a9e-4fcf-8468-6cd18c64a8b8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7454f83e-8a9e-4fcf-8468-6cd18c64a8b8-plugins-config-volume\") pod \"7454f83e-8a9e-4fcf-8468-6cd18c64a8b8\" (UID: \"7454f83e-8a9e-4fcf-8468-6cd18c64a8b8\") "
Oct 30 19:01:53.481744 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:53.481678    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7454f83e-8a9e-4fcf-8468-6cd18c64a8b8-plugins-config-volume" pod "7454f83e-8a9e-4fcf-8468-6cd18c64a8b8" (UID: "7454f83e-8a9e-4fcf-8468-6cd18c64a8b8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:01:53.481986 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:53.481847    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7454f83e-8a9e-4fcf-8468-6cd18c64a8b8-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:01:53.483390 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:53.483368    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6329d3ce-2871-4e1f-b557-399ccadd7d0f-kube-api-access-sghzs" pod "6329d3ce-2871-4e1f-b557-399ccadd7d0f" (UID: "6329d3ce-2871-4e1f-b557-399ccadd7d0f"). InnerVolumeSpecName "kube-api-access-sghzs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:53.483390 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:53.483379    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7454f83e-8a9e-4fcf-8468-6cd18c64a8b8-kube-api-access-hj28b" pod "7454f83e-8a9e-4fcf-8468-6cd18c64a8b8" (UID: "7454f83e-8a9e-4fcf-8468-6cd18c64a8b8"). InnerVolumeSpecName "kube-api-access-hj28b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:53.582983 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:53.582936    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sghzs\" (UniqueName: \"kubernetes.io/projected/6329d3ce-2871-4e1f-b557-399ccadd7d0f-kube-api-access-sghzs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:01:53.582983 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:53.582983    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hj28b\" (UniqueName: \"kubernetes.io/projected/7454f83e-8a9e-4fcf-8468-6cd18c64a8b8-kube-api-access-hj28b\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:01:54.118228 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:54.118198    5820 generic.go:356] "Generic (PLEG): container finished" podID="6329d3ce-2871-4e1f-b557-399ccadd7d0f" containerID="69e9ccfaa3b0b6a707b45b04c509b079de4e8ef812baad4e2d439e7f73a255f9" exitCode=137
Oct 30 19:01:54.118611 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:54.118245    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7112-8c3e32"
Oct 30 19:01:54.118611 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:54.118293    5820 scope.go:122] "RemoveContainer" containerID="69e9ccfaa3b0b6a707b45b04c509b079de4e8ef812baad4e2d439e7f73a255f9"
Oct 30 19:01:54.119910 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:54.119891    5820 generic.go:356] "Generic (PLEG): container finished" podID="7454f83e-8a9e-4fcf-8468-6cd18c64a8b8" containerID="58addef4eef7e5b1c52c2bd6b8be99dd153528b97eae32cd0ffd3cec10c0b9ba" exitCode=0
Oct 30 19:01:54.120026 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:54.119969    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r" event={"ID":"7454f83e-8a9e-4fcf-8468-6cd18c64a8b8","Type":"ContainerDied","Data":"58addef4eef7e5b1c52c2bd6b8be99dd153528b97eae32cd0ffd3cec10c0b9ba"}
Oct 30 19:01:54.120026 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:54.119983    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r"
Oct 30 19:01:54.120138 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:54.120038    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r" event={"ID":"7454f83e-8a9e-4fcf-8468-6cd18c64a8b8","Type":"ContainerDied","Data":"d97540625d74d6ea20e6d99f0e0c10bf0c181a8c2dfd629e1c26f9c257d8409e"}
Oct 30 19:01:54.128802 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:54.128784    5820 scope.go:122] "RemoveContainer" containerID="69e9ccfaa3b0b6a707b45b04c509b079de4e8ef812baad4e2d439e7f73a255f9"
Oct 30 19:01:54.129351 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:01:54.129253    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69e9ccfaa3b0b6a707b45b04c509b079de4e8ef812baad4e2d439e7f73a255f9\": container with ID starting with 69e9ccfaa3b0b6a707b45b04c509b079de4e8ef812baad4e2d439e7f73a255f9 not found: ID does not exist" containerID="69e9ccfaa3b0b6a707b45b04c509b079de4e8ef812baad4e2d439e7f73a255f9"
Oct 30 19:01:54.129351 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:54.129307    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69e9ccfaa3b0b6a707b45b04c509b079de4e8ef812baad4e2d439e7f73a255f9"} err="failed to get container status \"69e9ccfaa3b0b6a707b45b04c509b079de4e8ef812baad4e2d439e7f73a255f9\": rpc error: code = NotFound desc = could not find container \"69e9ccfaa3b0b6a707b45b04c509b079de4e8ef812baad4e2d439e7f73a255f9\": container with ID starting with 69e9ccfaa3b0b6a707b45b04c509b079de4e8ef812baad4e2d439e7f73a255f9 not found: ID does not exist"
Oct 30 19:01:54.129351 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:54.129330    5820 scope.go:122] "RemoveContainer" containerID="58addef4eef7e5b1c52c2bd6b8be99dd153528b97eae32cd0ffd3cec10c0b9ba"
Oct 30 19:01:54.139356 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:54.139339    5820 scope.go:122] "RemoveContainer" containerID="58addef4eef7e5b1c52c2bd6b8be99dd153528b97eae32cd0ffd3cec10c0b9ba"
Oct 30 19:01:54.139613 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:01:54.139595    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58addef4eef7e5b1c52c2bd6b8be99dd153528b97eae32cd0ffd3cec10c0b9ba\": container with ID starting with 58addef4eef7e5b1c52c2bd6b8be99dd153528b97eae32cd0ffd3cec10c0b9ba not found: ID does not exist" containerID="58addef4eef7e5b1c52c2bd6b8be99dd153528b97eae32cd0ffd3cec10c0b9ba"
Oct 30 19:01:54.139677 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:54.139620    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58addef4eef7e5b1c52c2bd6b8be99dd153528b97eae32cd0ffd3cec10c0b9ba"} err="failed to get container status \"58addef4eef7e5b1c52c2bd6b8be99dd153528b97eae32cd0ffd3cec10c0b9ba\": rpc error: code = NotFound desc = could not find container \"58addef4eef7e5b1c52c2bd6b8be99dd153528b97eae32cd0ffd3cec10c0b9ba\": container with ID starting with 58addef4eef7e5b1c52c2bd6b8be99dd153528b97eae32cd0ffd3cec10c0b9ba not found: ID does not exist"
Oct 30 19:01:54.285584 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:54.285547    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r"]
Oct 30 19:01:54.335868 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:54.335833    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-470093-59c4f7f57d-xm29r"]
Oct 30 19:01:54.808472 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:54.804989    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:55.437332 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:55.437297    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6329d3ce-2871-4e1f-b557-399ccadd7d0f" path="/var/lib/kubelet/pods/6329d3ce-2871-4e1f-b557-399ccadd7d0f/volumes"
Oct 30 19:01:55.437709 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:55.437690    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7454f83e-8a9e-4fcf-8468-6cd18c64a8b8" path="/var/lib/kubelet/pods/7454f83e-8a9e-4fcf-8468-6cd18c64a8b8/volumes"
Oct 30 19:01:55.553331 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:55.553303    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:58.758248 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:58.758207    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:58.993570 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:58.993505    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:59.871667 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:01:59.871604    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:03.224737 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:03.224703    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:03.448767 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:03.448733    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:04.663290 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:04.663245    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:08.204061 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:02:08.204020    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:02:09.439048 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.438447    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:09.523071 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.523035    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"]
Oct 30 19:02:09.524095 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.524071    5820 cpu_manager.go:414] "containerMap: removing container" podUID="6329d3ce-2871-4e1f-b557-399ccadd7d0f" containerName="test-curl-7112-8c3e32"
Oct 30 19:02:09.524095 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.524095    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6329d3ce-2871-4e1f-b557-399ccadd7d0f" containerName="test-curl-7112-8c3e32"
Oct 30 19:02:09.524269 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.524114    5820 cpu_manager.go:414] "containerMap: removing container" podUID="7454f83e-8a9e-4fcf-8468-6cd18c64a8b8" containerName="payload-processing"
Oct 30 19:02:09.524269 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.524121    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7454f83e-8a9e-4fcf-8468-6cd18c64a8b8" containerName="payload-processing"
Oct 30 19:02:09.524383 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.524330    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="6329d3ce-2871-4e1f-b557-399ccadd7d0f" containerName="test-curl-7112-8c3e32"
Oct 30 19:02:09.524383 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.524341    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="7454f83e-8a9e-4fcf-8468-6cd18c64a8b8" containerName="payload-processing"
Oct 30 19:02:09.528160 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.528136    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.685533 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.685493    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sntvn\" (UniqueName: \"kubernetes.io/projected/e3353c6a-b313-45af-a000-d826722d1416-kube-api-access-sntvn\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.685824 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.685795    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-credential-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.685993 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.685855    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3353c6a-b313-45af-a000-d826722d1416-istio-token\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.686567 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.686094    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-istio-envoy\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.686567 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.686179    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-istio-data\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.686567 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.686212    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3353c6a-b313-45af-a000-d826722d1416-istio-podinfo\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.686567 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.686243    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-workload-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.686567 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.686291    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3353c6a-b313-45af-a000-d826722d1416-istiod-ca-cert\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.686567 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.686385    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-workload-certs\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.736859 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.736775    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"]
Oct 30 19:02:09.787600 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.787564    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sntvn\" (UniqueName: \"kubernetes.io/projected/e3353c6a-b313-45af-a000-d826722d1416-kube-api-access-sntvn\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.787785 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.787618    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-credential-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.787785 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.787646    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3353c6a-b313-45af-a000-d826722d1416-istio-token\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.787785 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.787711    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-istio-envoy\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.787785 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.787773    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-istio-data\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.788214 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.787802    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3353c6a-b313-45af-a000-d826722d1416-istio-podinfo\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.788214 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.787832    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-workload-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.788214 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.787882    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3353c6a-b313-45af-a000-d826722d1416-istiod-ca-cert\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.788214 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.788113    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-credential-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.788214 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.788113    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-workload-certs\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.788214 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.788194    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-istio-data\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.788512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.788313    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-workload-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.788512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.788353    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-workload-certs\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.788650 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.788633    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3353c6a-b313-45af-a000-d826722d1416-istiod-ca-cert\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.790897 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.790867    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-istio-envoy\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.791020 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.790996    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3353c6a-b313-45af-a000-d826722d1416-istio-podinfo\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.848547 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.848513    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3353c6a-b313-45af-a000-d826722d1416-istio-token\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:09.848709 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:09.848671    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sntvn\" (UniqueName: \"kubernetes.io/projected/e3353c6a-b313-45af-a000-d826722d1416-kube-api-access-sntvn\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:10.142267 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:10.142227    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:10.575676 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:10.575652    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"]
Oct 30 19:02:10.577180 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:02:10.577146    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3353c6a_b313_45af_a000_d826722d1416.slice/crio-e48637d610daef46f7564e8b8f3105abb27b7ceaadf6b88246ee3608623b3908 WatchSource:0}: Error finding container e48637d610daef46f7564e8b8f3105abb27b7ceaadf6b88246ee3608623b3908: Status 404 returned error can't find the container with id e48637d610daef46f7564e8b8f3105abb27b7ceaadf6b88246ee3608623b3908
Oct 30 19:02:10.579571 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:10.579536    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:02:10.579659 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:10.579603    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:02:10.579659 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:10.579631    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:02:11.270475 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:11.270437    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n" event={"ID":"e3353c6a-b313-45af-a000-d826722d1416","Type":"ContainerStarted","Data":"65d21ec774f53dcabc16d8eaca22aff792cfce7737d28fbe8ed8bc295e023da2"}
Oct 30 19:02:11.270659 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:11.270496    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n" event={"ID":"e3353c6a-b313-45af-a000-d826722d1416","Type":"ContainerStarted","Data":"e48637d610daef46f7564e8b8f3105abb27b7ceaadf6b88246ee3608623b3908"}
Oct 30 19:02:11.488878 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:11.488825    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n" podStartSLOduration=2.4888128370000002 podStartE2EDuration="2.488812837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:11.446736165 +0000 UTC m=+1652.158484862" watchObservedRunningTime="2026-10-30 19:02:11.488812837 +0000 UTC m=+1652.200561533"
Oct 30 19:02:12.145400 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:12.145366    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:12.146082 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:12.146056    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 19:02:12.146268 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:12.146241    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n" podUID="e3353c6a-b313-45af-a000-d826722d1416" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 19:02:13.159151 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:13.159112    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 19:02:13.159709 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:13.159180    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n" podUID="e3353c6a-b313-45af-a000-d826722d1416" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 19:02:14.153985 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:14.145318    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 19:02:14.153985 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:14.145382    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n" podUID="e3353c6a-b313-45af-a000-d826722d1416" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 19:02:14.449028 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:14.445635    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:15.154982 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:15.150050    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 19:02:15.154982 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:15.150113    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n" podUID="e3353c6a-b313-45af-a000-d826722d1416" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 19:02:16.144977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:16.143293    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 19:02:16.144977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:16.143353    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n" podUID="e3353c6a-b313-45af-a000-d826722d1416" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 19:02:17.148829 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:17.148794    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:17.149344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:17.149324    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:17.150378 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:17.150343    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:02:19.250230 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:19.250199    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"]
Oct 30 19:02:19.255550 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:19.255532    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"
Oct 30 19:02:19.344430 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:19.344389    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0408420-508d-475e-a6cb-85e9dfc06d4b-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj\" (UID: \"f0408420-508d-475e-a6cb-85e9dfc06d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"
Oct 30 19:02:19.344625 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:19.344492    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0408420-508d-475e-a6cb-85e9dfc06d4b-metrics-certs\") pod \"maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj\" (UID: \"f0408420-508d-475e-a6cb-85e9dfc06d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"
Oct 30 19:02:19.344625 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:19.344526    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0408420-508d-475e-a6cb-85e9dfc06d4b-maas-api-tls\") pod \"maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj\" (UID: \"f0408420-508d-475e-a6cb-85e9dfc06d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"
Oct 30 19:02:19.344823 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:19.344628    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6gh7\" (UniqueName: \"kubernetes.io/projected/f0408420-508d-475e-a6cb-85e9dfc06d4b-kube-api-access-x6gh7\") pod \"maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj\" (UID: \"f0408420-508d-475e-a6cb-85e9dfc06d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"
Oct 30 19:02:19.359412 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:19.359386    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-82305a-serving-cert\""
Oct 30 19:02:19.363463 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:19.363431    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"]
Oct 30 19:02:19.446176 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:19.446141    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0408420-508d-475e-a6cb-85e9dfc06d4b-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj\" (UID: \"f0408420-508d-475e-a6cb-85e9dfc06d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"
Oct 30 19:02:19.446336 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:19.446203    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0408420-508d-475e-a6cb-85e9dfc06d4b-metrics-certs\") pod \"maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj\" (UID: \"f0408420-508d-475e-a6cb-85e9dfc06d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"
Oct 30 19:02:19.446336 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:19.446253    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0408420-508d-475e-a6cb-85e9dfc06d4b-maas-api-tls\") pod \"maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj\" (UID: \"f0408420-508d-475e-a6cb-85e9dfc06d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"
Oct 30 19:02:19.446336 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:19.446325    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6gh7\" (UniqueName: \"kubernetes.io/projected/f0408420-508d-475e-a6cb-85e9dfc06d4b-kube-api-access-x6gh7\") pod \"maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj\" (UID: \"f0408420-508d-475e-a6cb-85e9dfc06d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"
Oct 30 19:02:19.446511 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:02:19.446489    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-serving-cert: secret "maas-api-e2e-ipp-a-82305a-serving-cert" not found
Oct 30 19:02:19.446584 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:02:19.446572    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0408420-508d-475e-a6cb-85e9dfc06d4b-maas-api-tls podName:f0408420-508d-475e-a6cb-85e9dfc06d4b nodeName:}" failed. No retries permitted until 2026-10-30 19:02:19.946553667 +0000 UTC m=+1660.658302359 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f0408420-508d-475e-a6cb-85e9dfc06d4b-maas-api-tls") pod "maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj" (UID: "f0408420-508d-475e-a6cb-85e9dfc06d4b") : secret "maas-api-e2e-ipp-a-82305a-serving-cert" not found
Oct 30 19:02:19.446869 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:19.446849    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0408420-508d-475e-a6cb-85e9dfc06d4b-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj\" (UID: \"f0408420-508d-475e-a6cb-85e9dfc06d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"
Oct 30 19:02:19.448782 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:19.448763    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0408420-508d-475e-a6cb-85e9dfc06d4b-metrics-certs\") pod \"maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj\" (UID: \"f0408420-508d-475e-a6cb-85e9dfc06d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"
Oct 30 19:02:19.519601 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:19.519448    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:19.545571 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:19.545524    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6gh7\" (UniqueName: \"kubernetes.io/projected/f0408420-508d-475e-a6cb-85e9dfc06d4b-kube-api-access-x6gh7\") pod \"maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj\" (UID: \"f0408420-508d-475e-a6cb-85e9dfc06d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"
Oct 30 19:02:19.715851 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:19.715816    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:19.915499 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:19.915453    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:19.955792 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:19.955766    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0408420-508d-475e-a6cb-85e9dfc06d4b-maas-api-tls\") pod \"maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj\" (UID: \"f0408420-508d-475e-a6cb-85e9dfc06d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"
Oct 30 19:02:19.958329 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:19.958304    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0408420-508d-475e-a6cb-85e9dfc06d4b-maas-api-tls\") pod \"maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj\" (UID: \"f0408420-508d-475e-a6cb-85e9dfc06d4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"
Oct 30 19:02:20.171110 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:20.171026    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"
Oct 30 19:02:20.552496 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:20.552466    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"]
Oct 30 19:02:20.554152 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:02:20.554127    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0408420_508d_475e_a6cb_85e9dfc06d4b.slice/crio-8b9ab48cad96d0f1a6788c2f7372d4431b95cb78e035909bcb9b58ec0b1e5114 WatchSource:0}: Error finding container 8b9ab48cad96d0f1a6788c2f7372d4431b95cb78e035909bcb9b58ec0b1e5114: Status 404 returned error can't find the container with id 8b9ab48cad96d0f1a6788c2f7372d4431b95cb78e035909bcb9b58ec0b1e5114
Oct 30 19:02:21.374837 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:21.374800    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj" event={"ID":"f0408420-508d-475e-a6cb-85e9dfc06d4b","Type":"ContainerStarted","Data":"8b9ab48cad96d0f1a6788c2f7372d4431b95cb78e035909bcb9b58ec0b1e5114"}
Oct 30 19:02:22.413204 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:22.413163    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj" event={"ID":"f0408420-508d-475e-a6cb-85e9dfc06d4b","Type":"ContainerStarted","Data":"b182c40ea8a5e7ba9a8be064d80a5d88f0c677a0722d861224902b9c1ea73853"}
Oct 30 19:02:22.414243 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:22.414214    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"
Oct 30 19:02:22.532903 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:22.532851    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj" podStartSLOduration=2.572149895 podStartE2EDuration="3.532833342s" totalImagesPullingTime="960.683447ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:22.532232172 +0000 UTC m=+1663.243980870" watchObservedRunningTime="2026-10-30 19:02:22.532833342 +0000 UTC m=+1663.244582040"
Oct 30 19:02:23.738818 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:23.738775    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"]
Oct 30 19:02:23.739278 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:23.739040    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr" podUID="7d7e45b3-4b0d-4789-b6be-17f080656583" containerName="maas-api" containerID="cri-o://022ae211234b0ed1e28b2d97caf594b378e88339a2b9bd15625906e6b9a36501" gracePeriod=30
Oct 30 19:02:23.871439 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:23.871398    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:24.124311 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.123976    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"
Oct 30 19:02:24.241579 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.240819    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d7e45b3-4b0d-4789-b6be-17f080656583-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d7e45b3-4b0d-4789-b6be-17f080656583-openshift-service-ca\") pod \"7d7e45b3-4b0d-4789-b6be-17f080656583\" (UID: \"7d7e45b3-4b0d-4789-b6be-17f080656583\") "
Oct 30 19:02:24.241579 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.241027    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d7e45b3-4b0d-4789-b6be-17f080656583-kube-api-access-926m8\" (UniqueName: \"kubernetes.io/projected/7d7e45b3-4b0d-4789-b6be-17f080656583-kube-api-access-926m8\") pod \"7d7e45b3-4b0d-4789-b6be-17f080656583\" (UID: \"7d7e45b3-4b0d-4789-b6be-17f080656583\") "
Oct 30 19:02:24.241579 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.241115    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7d7e45b3-4b0d-4789-b6be-17f080656583-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d7e45b3-4b0d-4789-b6be-17f080656583-metrics-certs\") pod \"7d7e45b3-4b0d-4789-b6be-17f080656583\" (UID: \"7d7e45b3-4b0d-4789-b6be-17f080656583\") "
Oct 30 19:02:24.241579 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.241186    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7d7e45b3-4b0d-4789-b6be-17f080656583-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d7e45b3-4b0d-4789-b6be-17f080656583-maas-api-tls\") pod \"7d7e45b3-4b0d-4789-b6be-17f080656583\" (UID: \"7d7e45b3-4b0d-4789-b6be-17f080656583\") "
Oct 30 19:02:24.268218 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.268179    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d7e45b3-4b0d-4789-b6be-17f080656583-kube-api-access-926m8" pod "7d7e45b3-4b0d-4789-b6be-17f080656583" (UID: "7d7e45b3-4b0d-4789-b6be-17f080656583"). InnerVolumeSpecName "kube-api-access-926m8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:02:24.272532 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.272498    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d7e45b3-4b0d-4789-b6be-17f080656583-openshift-service-ca" pod "7d7e45b3-4b0d-4789-b6be-17f080656583" (UID: "7d7e45b3-4b0d-4789-b6be-17f080656583"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:02:24.292036 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.282393    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d7e45b3-4b0d-4789-b6be-17f080656583-maas-api-tls" pod "7d7e45b3-4b0d-4789-b6be-17f080656583" (UID: "7d7e45b3-4b0d-4789-b6be-17f080656583"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:02:24.292523 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.292494    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d7e45b3-4b0d-4789-b6be-17f080656583-metrics-certs" pod "7d7e45b3-4b0d-4789-b6be-17f080656583" (UID: "7d7e45b3-4b0d-4789-b6be-17f080656583"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:02:24.344919 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.344837    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d7e45b3-4b0d-4789-b6be-17f080656583-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:24.344919 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.344869    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d7e45b3-4b0d-4789-b6be-17f080656583-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:24.344919 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.344878    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-926m8\" (UniqueName: \"kubernetes.io/projected/7d7e45b3-4b0d-4789-b6be-17f080656583-kube-api-access-926m8\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:24.344919 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.344887    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d7e45b3-4b0d-4789-b6be-17f080656583-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:24.428325 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.428289    5820 generic.go:356] "Generic (PLEG): container finished" podID="7d7e45b3-4b0d-4789-b6be-17f080656583" containerID="022ae211234b0ed1e28b2d97caf594b378e88339a2b9bd15625906e6b9a36501" exitCode=0
Oct 30 19:02:24.428496 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.428329    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"
Oct 30 19:02:24.428496 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.428381    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr" event={"ID":"7d7e45b3-4b0d-4789-b6be-17f080656583","Type":"ContainerDied","Data":"022ae211234b0ed1e28b2d97caf594b378e88339a2b9bd15625906e6b9a36501"}
Oct 30 19:02:24.428496 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.428430    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr" event={"ID":"7d7e45b3-4b0d-4789-b6be-17f080656583","Type":"ContainerDied","Data":"aa4afeeb4c614765a04959eef67574649c0fff1f74c0bdb4014fb09e577072dd"}
Oct 30 19:02:24.428496 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.428454    5820 scope.go:122] "RemoveContainer" containerID="022ae211234b0ed1e28b2d97caf594b378e88339a2b9bd15625906e6b9a36501"
Oct 30 19:02:24.439275 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.439255    5820 scope.go:122] "RemoveContainer" containerID="022ae211234b0ed1e28b2d97caf594b378e88339a2b9bd15625906e6b9a36501"
Oct 30 19:02:24.439563 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:02:24.439536    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"022ae211234b0ed1e28b2d97caf594b378e88339a2b9bd15625906e6b9a36501\": container with ID starting with 022ae211234b0ed1e28b2d97caf594b378e88339a2b9bd15625906e6b9a36501 not found: ID does not exist" containerID="022ae211234b0ed1e28b2d97caf594b378e88339a2b9bd15625906e6b9a36501"
Oct 30 19:02:24.439652 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.439567    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"022ae211234b0ed1e28b2d97caf594b378e88339a2b9bd15625906e6b9a36501"} err="failed to get container status \"022ae211234b0ed1e28b2d97caf594b378e88339a2b9bd15625906e6b9a36501\": rpc error: code = NotFound desc = could not find container \"022ae211234b0ed1e28b2d97caf594b378e88339a2b9bd15625906e6b9a36501\": container with ID starting with 022ae211234b0ed1e28b2d97caf594b378e88339a2b9bd15625906e6b9a36501 not found: ID does not exist"
Oct 30 19:02:24.596365 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.596295    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"]
Oct 30 19:02:24.601502 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.601478    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-66baf45c-fc7b65c85-nzpfr"]
Oct 30 19:02:24.643140 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.643109    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"]
Oct 30 19:02:24.643859 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.643822    5820 cpu_manager.go:414] "containerMap: removing container" podUID="7d7e45b3-4b0d-4789-b6be-17f080656583" containerName="maas-api"
Oct 30 19:02:24.643859 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.643845    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d7e45b3-4b0d-4789-b6be-17f080656583" containerName="maas-api"
Oct 30 19:02:24.644092 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.643995    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d7e45b3-4b0d-4789-b6be-17f080656583" containerName="maas-api"
Oct 30 19:02:24.648829 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.648809    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.758812 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.758772    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-workload-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.795664 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.795627    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-data\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.833979 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.799162    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-envoy\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.834357 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.834320    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-workload-certs\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.834607 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.834589    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istiod-ca-cert\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.834785 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.834769    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-credential-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.834897 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.834882    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-token\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.835090 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.835073    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvw5s\" (UniqueName: \"kubernetes.io/projected/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-kube-api-access-wvw5s\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.847755 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.847677    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-podinfo\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.898442 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.898402    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"]
Oct 30 19:02:24.959391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.950724    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-workload-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.959391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.950822    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-data\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.959391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.951349    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-data\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.959391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.951381    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-workload-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.959391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.951570    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-envoy\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.959391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.951618    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-workload-certs\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.959391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.951724    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istiod-ca-cert\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.959391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.951805    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-credential-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.959391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.951835    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-token\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.959391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.951914    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvw5s\" (UniqueName: \"kubernetes.io/projected/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-kube-api-access-wvw5s\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.959391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.952644    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istiod-ca-cert\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.959391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.952683    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-credential-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.959391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.952691    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-podinfo\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.959391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.953283    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-workload-certs\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.982921 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.979633    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-podinfo\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:24.982921 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:24.980068    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-envoy\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:25.016361 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:25.016334    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-token\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:25.016548 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:25.016526    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvw5s\" (UniqueName: \"kubernetes.io/projected/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-kube-api-access-wvw5s\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:25.061869 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:25.061835    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:25.300028 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:25.299158    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:25.470859 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:25.470725    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:25.481908 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:25.481875    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d7e45b3-4b0d-4789-b6be-17f080656583" path="/var/lib/kubelet/pods/7d7e45b3-4b0d-4789-b6be-17f080656583/volumes"
Oct 30 19:02:25.738052 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:25.724319    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:25.882515 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:25.882485    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"]
Oct 30 19:02:25.884397 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:02:25.884360    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d39a7cd_562c_49cc_9ce2_63e30d53eae0.slice/crio-d72a8c5df88c95fe14a2124b3dc3e4cf5dd638da8b931f4d1d7396e1d5c14e26 WatchSource:0}: Error finding container d72a8c5df88c95fe14a2124b3dc3e4cf5dd638da8b931f4d1d7396e1d5c14e26: Status 404 returned error can't find the container with id d72a8c5df88c95fe14a2124b3dc3e4cf5dd638da8b931f4d1d7396e1d5c14e26
Oct 30 19:02:25.887383 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:25.887351    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:02:25.887503 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:25.887415    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:02:25.887503 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:25.887443    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:02:26.421187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:26.421088    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:26.509550 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:26.509512    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8" event={"ID":"8d39a7cd-562c-49cc-9ce2-63e30d53eae0","Type":"ContainerStarted","Data":"47c250b9e9e1786910a967902260af6afcbcb6aec9f0ac84bf401fe6682f6ccd"}
Oct 30 19:02:26.509823 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:26.509802    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8" event={"ID":"8d39a7cd-562c-49cc-9ce2-63e30d53eae0","Type":"ContainerStarted","Data":"d72a8c5df88c95fe14a2124b3dc3e4cf5dd638da8b931f4d1d7396e1d5c14e26"}
Oct 30 19:02:26.677031 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:26.676911    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:26.734018 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:26.719214    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8" podStartSLOduration=2.719191772 podStartE2EDuration="2.719191772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:26.683276305 +0000 UTC m=+1667.395025004" watchObservedRunningTime="2026-10-30 19:02:26.719191772 +0000 UTC m=+1667.430940472"
Oct 30 19:02:26.963025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:26.960704    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:27.162389 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:27.162352    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:27.299678 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:27.299649    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:27.301232 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:27.301199    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 30 19:02:27.301362 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:27.301247    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8" podUID="8d39a7cd-562c-49cc-9ce2-63e30d53eae0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 30 19:02:27.681925 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:27.681836    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:28.300668 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:28.300630    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 30 19:02:28.301194 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:28.300693    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8" podUID="8d39a7cd-562c-49cc-9ce2-63e30d53eae0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 30 19:02:28.426556 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:28.426521    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"
Oct 30 19:02:29.300414 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:29.300380    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 30 19:02:29.300616 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:29.300437    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8" podUID="8d39a7cd-562c-49cc-9ce2-63e30d53eae0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 30 19:02:30.300401 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:30.300354    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 30 19:02:30.300796 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:30.300444    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8" podUID="8d39a7cd-562c-49cc-9ce2-63e30d53eae0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 30 19:02:31.300816 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:31.300779    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 30 19:02:31.301264 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:31.300833    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8" podUID="8d39a7cd-562c-49cc-9ce2-63e30d53eae0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 30 19:02:31.462980 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:31.462867    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:32.186393 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:32.186356    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:32.308156 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:32.308118    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 30 19:02:32.310507 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:32.310470    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8" podUID="8d39a7cd-562c-49cc-9ce2-63e30d53eae0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 30 19:02:33.304673 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:33.304640    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:33.305017 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:33.304997    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:33.305705 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:33.305689    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:02:33.981584 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:33.981552    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:34.219341 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:34.219301    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:34.407468 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:34.407433    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"]
Oct 30 19:02:34.407690 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:34.407669    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l" podUID="d2844bd5-87c5-4f3c-86d4-d4041c484355" containerName="istio-proxy" containerID="cri-o://f6f79e29a8155849464356ea5327e87a2acd0e7c4ab4615086112f7cafda5e5d" gracePeriod=30
Oct 30 19:02:34.447982 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:34.447916    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:37.159927 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:37.151833    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:38.205091 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:02:38.205059    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:02:39.671454 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.671425    5820 generic.go:356] "Generic (PLEG): container finished" podID="d2844bd5-87c5-4f3c-86d4-d4041c484355" containerID="f6f79e29a8155849464356ea5327e87a2acd0e7c4ab4615086112f7cafda5e5d" exitCode=0
Oct 30 19:02:39.671926 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.671496    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l" event={"ID":"d2844bd5-87c5-4f3c-86d4-d4041c484355","Type":"ContainerDied","Data":"f6f79e29a8155849464356ea5327e87a2acd0e7c4ab4615086112f7cafda5e5d"}
Oct 30 19:02:39.671926 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.671584    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l" event={"ID":"d2844bd5-87c5-4f3c-86d4-d4041c484355","Type":"ContainerDied","Data":"8fd7212cddc8345454b544be3467490236f5df8f24c0545755d9cd3e362fcf62"}
Oct 30 19:02:39.671926 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.671613    5820 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8fd7212cddc8345454b544be3467490236f5df8f24c0545755d9cd3e362fcf62"
Oct 30 19:02:39.671926 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.671632    5820 scope.go:122] "RemoveContainer" containerID="438254d3c23b54711498fc918c159a8bb50f7b6275335c54032d451aa907dced"
Oct 30 19:02:39.694711 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:02:39.694678    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"438254d3c23b54711498fc918c159a8bb50f7b6275335c54032d451aa907dced\": container with ID starting with 438254d3c23b54711498fc918c159a8bb50f7b6275335c54032d451aa907dced not found: ID does not exist" containerID="438254d3c23b54711498fc918c159a8bb50f7b6275335c54032d451aa907dced"
Oct 30 19:02:39.698254 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.698232    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 19:02:39.847784 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.847759    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-workload-certs\") pod \"d2844bd5-87c5-4f3c-86d4-d4041c484355\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") "
Oct 30 19:02:39.847903 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.847804    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-data\") pod \"d2844bd5-87c5-4f3c-86d4-d4041c484355\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") "
Oct 30 19:02:39.847903 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.847867    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-envoy\") pod \"d2844bd5-87c5-4f3c-86d4-d4041c484355\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") "
Oct 30 19:02:39.848044 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.847940    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-podinfo\") pod \"d2844bd5-87c5-4f3c-86d4-d4041c484355\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") "
Oct 30 19:02:39.848044 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.848036    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d2844bd5-87c5-4f3c-86d4-d4041c484355-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2844bd5-87c5-4f3c-86d4-d4041c484355-istiod-ca-cert\") pod \"d2844bd5-87c5-4f3c-86d4-d4041c484355\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") "
Oct 30 19:02:39.848161 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.848067    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-workload-certs" pod "d2844bd5-87c5-4f3c-86d4-d4041c484355" (UID: "d2844bd5-87c5-4f3c-86d4-d4041c484355"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:39.848161 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.848077    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-workload-socket\") pod \"d2844bd5-87c5-4f3c-86d4-d4041c484355\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") "
Oct 30 19:02:39.848161 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.848134    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-data" pod "d2844bd5-87c5-4f3c-86d4-d4041c484355" (UID: "d2844bd5-87c5-4f3c-86d4-d4041c484355"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:39.848324 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.848176    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-token\" (UniqueName: \"kubernetes.io/projected/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-token\") pod \"d2844bd5-87c5-4f3c-86d4-d4041c484355\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") "
Oct 30 19:02:39.848324 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.848231    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d2844bd5-87c5-4f3c-86d4-d4041c484355-kube-api-access-bvn5p\" (UniqueName: \"kubernetes.io/projected/d2844bd5-87c5-4f3c-86d4-d4041c484355-kube-api-access-bvn5p\") pod \"d2844bd5-87c5-4f3c-86d4-d4041c484355\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") "
Oct 30 19:02:39.848324 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.848290    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-workload-socket" pod "d2844bd5-87c5-4f3c-86d4-d4041c484355" (UID: "d2844bd5-87c5-4f3c-86d4-d4041c484355"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:39.848492 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.848369    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-credential-socket\") pod \"d2844bd5-87c5-4f3c-86d4-d4041c484355\" (UID: \"d2844bd5-87c5-4f3c-86d4-d4041c484355\") "
Oct 30 19:02:39.848607 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.848586    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2844bd5-87c5-4f3c-86d4-d4041c484355-istiod-ca-cert" pod "d2844bd5-87c5-4f3c-86d4-d4041c484355" (UID: "d2844bd5-87c5-4f3c-86d4-d4041c484355"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:02:39.848926 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.848805    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-credential-socket" pod "d2844bd5-87c5-4f3c-86d4-d4041c484355" (UID: "d2844bd5-87c5-4f3c-86d4-d4041c484355"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:39.849305 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.849281    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:39.849456 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.849440    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:39.849588 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.849567    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2844bd5-87c5-4f3c-86d4-d4041c484355-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:39.849735 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.849593    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:39.849735 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.849609    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:39.850743 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.850714    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-podinfo" pod "d2844bd5-87c5-4f3c-86d4-d4041c484355" (UID: "d2844bd5-87c5-4f3c-86d4-d4041c484355"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:02:39.850845 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.850794    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-envoy" pod "d2844bd5-87c5-4f3c-86d4-d4041c484355" (UID: "d2844bd5-87c5-4f3c-86d4-d4041c484355"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:39.851139 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.851106    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-token" pod "d2844bd5-87c5-4f3c-86d4-d4041c484355" (UID: "d2844bd5-87c5-4f3c-86d4-d4041c484355"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:02:39.851244 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.851197    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2844bd5-87c5-4f3c-86d4-d4041c484355-kube-api-access-bvn5p" pod "d2844bd5-87c5-4f3c-86d4-d4041c484355" (UID: "d2844bd5-87c5-4f3c-86d4-d4041c484355"). InnerVolumeSpecName "kube-api-access-bvn5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:02:39.950607 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.950541    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvn5p\" (UniqueName: \"kubernetes.io/projected/d2844bd5-87c5-4f3c-86d4-d4041c484355-kube-api-access-bvn5p\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:39.950607 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.950567    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:39.950607 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.950576    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:39.950607 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:39.950586    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2844bd5-87c5-4f3c-86d4-d4041c484355-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:40.680397 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:40.680357    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"
Oct 30 19:02:40.896463 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:40.896432    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"]
Oct 30 19:02:40.945896 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:40.945816    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-85ab6a-openshift-default-c747dfdf9-j6j6l"]
Oct 30 19:02:41.437918 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:41.437884    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d2844bd5-87c5-4f3c-86d4-d4041c484355" path="/var/lib/kubelet/pods/d2844bd5-87c5-4f3c-86d4-d4041c484355/volumes"
Oct 30 19:02:42.086974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:42.086198    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:43.782944 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:43.782907    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"]
Oct 30 19:02:43.783419 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:43.783177    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6" podUID="4be0753d-602a-4f4a-a02d-5d6baf686df2" containerName="istio-proxy" containerID="cri-o://157cc81bc9ef32c0907cd568640071754e0533cf660713eeeca407afa24f63da" gracePeriod=30
Oct 30 19:02:43.832278 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:43.832250    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"]
Oct 30 19:02:43.832524 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:43.832497    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv" podUID="98a5a772-0770-40c9-8a01-5b991fee55d4" containerName="istio-proxy" containerID="cri-o://8ed8e4f60fd86b64e0d314cb04a00444d1256f43c49341a643981caf79ccee67" gracePeriod=30
Oct 30 19:02:43.835326 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:43.835303    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"]
Oct 30 19:02:43.835536 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:43.835501    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms" podUID="fcef9b8a-2a3a-4e51-875d-131d764e1ed0" containerName="istio-proxy" containerID="cri-o://751f68cb8d9b7adf06705351fa4d4a2eaaaaac525c08300a50bf84c93f0b0076" gracePeriod=30
Oct 30 19:02:43.882883 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:43.882855    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"]
Oct 30 19:02:43.883082 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:43.883046    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8" podUID="562448ab-2cf6-4fe5-b2f0-0c227a4ef70a" containerName="istio-proxy" containerID="cri-o://dc2babf0a5be41fcba7e033b1f2208215296bf988cb598b0ca9ff6fa72579e1a" gracePeriod=30
Oct 30 19:02:46.808960 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:46.808920    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv"]
Oct 30 19:02:47.188310 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:47.188219    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:47.399016 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:47.398980    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:48.247335 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:48.247301    5820 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:02:48.247725 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:48.247357    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv" podUID="98a5a772-0770-40c9-8a01-5b991fee55d4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:02:49.146756 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.146420    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 19:02:49.193069 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.193026    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 19:02:49.264416 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.264388    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4be0753d-602a-4f4a-a02d-5d6baf686df2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4be0753d-602a-4f4a-a02d-5d6baf686df2-istiod-ca-cert\") pod \"4be0753d-602a-4f4a-a02d-5d6baf686df2\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") "
Oct 30 19:02:49.264877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.264429    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-data\") pod \"4be0753d-602a-4f4a-a02d-5d6baf686df2\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") "
Oct 30 19:02:49.264877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.264572    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-workload-certs\") pod \"4be0753d-602a-4f4a-a02d-5d6baf686df2\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") "
Oct 30 19:02:49.264877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.264654    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-podinfo\") pod \"4be0753d-602a-4f4a-a02d-5d6baf686df2\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") "
Oct 30 19:02:49.264877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.264694    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-workload-socket\") pod \"4be0753d-602a-4f4a-a02d-5d6baf686df2\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") "
Oct 30 19:02:49.264877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.264782    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4be0753d-602a-4f4a-a02d-5d6baf686df2-istiod-ca-cert" pod "4be0753d-602a-4f4a-a02d-5d6baf686df2" (UID: "4be0753d-602a-4f4a-a02d-5d6baf686df2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:02:49.264877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.264796    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4be0753d-602a-4f4a-a02d-5d6baf686df2-kube-api-access-qxzlk\" (UniqueName: \"kubernetes.io/projected/4be0753d-602a-4f4a-a02d-5d6baf686df2-kube-api-access-qxzlk\") pod \"4be0753d-602a-4f4a-a02d-5d6baf686df2\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") "
Oct 30 19:02:49.264877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.264817    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-data" pod "4be0753d-602a-4f4a-a02d-5d6baf686df2" (UID: "4be0753d-602a-4f4a-a02d-5d6baf686df2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.264877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.264880    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-token\" (UniqueName: \"kubernetes.io/projected/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-token\") pod \"4be0753d-602a-4f4a-a02d-5d6baf686df2\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") "
Oct 30 19:02:49.265441 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.264888    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-workload-certs" pod "4be0753d-602a-4f4a-a02d-5d6baf686df2" (UID: "4be0753d-602a-4f4a-a02d-5d6baf686df2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.265441 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.264910    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-envoy\") pod \"4be0753d-602a-4f4a-a02d-5d6baf686df2\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") "
Oct 30 19:02:49.265441 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.264963    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-credential-socket\") pod \"4be0753d-602a-4f4a-a02d-5d6baf686df2\" (UID: \"4be0753d-602a-4f4a-a02d-5d6baf686df2\") "
Oct 30 19:02:49.265441 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.265076    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-workload-socket" pod "4be0753d-602a-4f4a-a02d-5d6baf686df2" (UID: "4be0753d-602a-4f4a-a02d-5d6baf686df2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.265703 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.265586    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-credential-socket" pod "4be0753d-602a-4f4a-a02d-5d6baf686df2" (UID: "4be0753d-602a-4f4a-a02d-5d6baf686df2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.265703 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.265642    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4be0753d-602a-4f4a-a02d-5d6baf686df2-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.265703 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.265663    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.265703 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.265675    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.265703 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.265686    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.267424 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.267402    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4be0753d-602a-4f4a-a02d-5d6baf686df2-kube-api-access-qxzlk" pod "4be0753d-602a-4f4a-a02d-5d6baf686df2" (UID: "4be0753d-602a-4f4a-a02d-5d6baf686df2"). InnerVolumeSpecName "kube-api-access-qxzlk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:02:49.267601 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.267582    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-podinfo" pod "4be0753d-602a-4f4a-a02d-5d6baf686df2" (UID: "4be0753d-602a-4f4a-a02d-5d6baf686df2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:02:49.267671 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.267585    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-token" pod "4be0753d-602a-4f4a-a02d-5d6baf686df2" (UID: "4be0753d-602a-4f4a-a02d-5d6baf686df2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:02:49.267924 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.267895    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-envoy" pod "4be0753d-602a-4f4a-a02d-5d6baf686df2" (UID: "4be0753d-602a-4f4a-a02d-5d6baf686df2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.270297 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.270274    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 19:02:49.282312 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.282293    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 19:02:49.366341 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366242    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-credential-socket\") pod \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") "
Oct 30 19:02:49.366341 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366296    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-envoy\") pod \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") "
Oct 30 19:02:49.366341 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366320    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-podinfo\") pod \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") "
Oct 30 19:02:49.366341 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366343    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-envoy\") pod \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") "
Oct 30 19:02:49.366691 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366382    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-token\" (UniqueName: \"kubernetes.io/projected/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-token\") pod \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") "
Oct 30 19:02:49.366691 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366408    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-workload-socket\") pod \"98a5a772-0770-40c9-8a01-5b991fee55d4\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") "
Oct 30 19:02:49.366691 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366429    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-workload-certs\") pod \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") "
Oct 30 19:02:49.366691 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366473    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-workload-socket\") pod \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") "
Oct 30 19:02:49.366691 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366522    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-token\" (UniqueName: \"kubernetes.io/projected/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-token\") pod \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") "
Oct 30 19:02:49.366691 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366559    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-envoy\") pod \"98a5a772-0770-40c9-8a01-5b991fee55d4\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") "
Oct 30 19:02:49.366691 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366589    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-workload-certs\") pod \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") "
Oct 30 19:02:49.366691 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366619    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istiod-ca-cert\") pod \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") "
Oct 30 19:02:49.366691 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366657    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-data\") pod \"98a5a772-0770-40c9-8a01-5b991fee55d4\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") "
Oct 30 19:02:49.366691 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366684    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98a5a772-0770-40c9-8a01-5b991fee55d4-kube-api-access-ffn62\" (UniqueName: \"kubernetes.io/projected/98a5a772-0770-40c9-8a01-5b991fee55d4-kube-api-access-ffn62\") pod \"98a5a772-0770-40c9-8a01-5b991fee55d4\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") "
Oct 30 19:02:49.367266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366712    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-podinfo\") pod \"98a5a772-0770-40c9-8a01-5b991fee55d4\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") "
Oct 30 19:02:49.367266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366743    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-credential-socket\") pod \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") "
Oct 30 19:02:49.367266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366773    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-data\") pod \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") "
Oct 30 19:02:49.367266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366814    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-workload-socket\") pod \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") "
Oct 30 19:02:49.367266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366833    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-podinfo\") pod \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") "
Oct 30 19:02:49.367266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366852    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-kube-api-access-f8q7b\" (UniqueName: \"kubernetes.io/projected/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-kube-api-access-f8q7b\") pod \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") "
Oct 30 19:02:49.367266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366885    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istiod-ca-cert\") pod \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\" (UID: \"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a\") "
Oct 30 19:02:49.367266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366970    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-workload-certs\") pod \"98a5a772-0770-40c9-8a01-5b991fee55d4\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") "
Oct 30 19:02:49.367266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.367005    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-kube-api-access-gljr4\" (UniqueName: \"kubernetes.io/projected/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-kube-api-access-gljr4\") pod \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") "
Oct 30 19:02:49.367266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.367041    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-token\" (UniqueName: \"kubernetes.io/projected/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-token\") pod \"98a5a772-0770-40c9-8a01-5b991fee55d4\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") "
Oct 30 19:02:49.367266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.367065    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/98a5a772-0770-40c9-8a01-5b991fee55d4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98a5a772-0770-40c9-8a01-5b991fee55d4-istiod-ca-cert\") pod \"98a5a772-0770-40c9-8a01-5b991fee55d4\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") "
Oct 30 19:02:49.367266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.367138    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-data\") pod \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\" (UID: \"fcef9b8a-2a3a-4e51-875d-131d764e1ed0\") "
Oct 30 19:02:49.367266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.367229    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-credential-socket\") pod \"98a5a772-0770-40c9-8a01-5b991fee55d4\" (UID: \"98a5a772-0770-40c9-8a01-5b991fee55d4\") "
Oct 30 19:02:49.369628 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366563    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-credential-socket" pod "562448ab-2cf6-4fe5-b2f0-0c227a4ef70a" (UID: "562448ab-2cf6-4fe5-b2f0-0c227a4ef70a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.369628 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.366748    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-workload-socket" pod "98a5a772-0770-40c9-8a01-5b991fee55d4" (UID: "98a5a772-0770-40c9-8a01-5b991fee55d4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.369628 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.368069    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-workload-socket" pod "562448ab-2cf6-4fe5-b2f0-0c227a4ef70a" (UID: "562448ab-2cf6-4fe5-b2f0-0c227a4ef70a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.369628 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.368314    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.369628 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.368338    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.369628 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.368332    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-workload-certs" pod "fcef9b8a-2a3a-4e51-875d-131d764e1ed0" (UID: "fcef9b8a-2a3a-4e51-875d-131d764e1ed0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.369628 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.368357    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxzlk\" (UniqueName: \"kubernetes.io/projected/4be0753d-602a-4f4a-a02d-5d6baf686df2-kube-api-access-qxzlk\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.369628 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.368382    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.369628 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.368397    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.369628 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.368411    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4be0753d-602a-4f4a-a02d-5d6baf686df2-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.369628 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.368425    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.369628 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.368439    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.369628 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.368927    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-workload-certs" pod "562448ab-2cf6-4fe5-b2f0-0c227a4ef70a" (UID: "562448ab-2cf6-4fe5-b2f0-0c227a4ef70a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.369628 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.368968    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istiod-ca-cert" pod "fcef9b8a-2a3a-4e51-875d-131d764e1ed0" (UID: "fcef9b8a-2a3a-4e51-875d-131d764e1ed0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:02:49.370622 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.370593    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-workload-certs" pod "98a5a772-0770-40c9-8a01-5b991fee55d4" (UID: "98a5a772-0770-40c9-8a01-5b991fee55d4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.370742 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.370630    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-credential-socket" pod "fcef9b8a-2a3a-4e51-875d-131d764e1ed0" (UID: "fcef9b8a-2a3a-4e51-875d-131d764e1ed0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.371720 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.371532    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istiod-ca-cert" pod "562448ab-2cf6-4fe5-b2f0-0c227a4ef70a" (UID: "562448ab-2cf6-4fe5-b2f0-0c227a4ef70a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:02:49.371720 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.371608    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-envoy" pod "562448ab-2cf6-4fe5-b2f0-0c227a4ef70a" (UID: "562448ab-2cf6-4fe5-b2f0-0c227a4ef70a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.372289 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.371680    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-workload-socket" pod "fcef9b8a-2a3a-4e51-875d-131d764e1ed0" (UID: "fcef9b8a-2a3a-4e51-875d-131d764e1ed0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.372289 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.372207    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98a5a772-0770-40c9-8a01-5b991fee55d4-istiod-ca-cert" pod "98a5a772-0770-40c9-8a01-5b991fee55d4" (UID: "98a5a772-0770-40c9-8a01-5b991fee55d4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:02:49.372564 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.372480    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-data" pod "fcef9b8a-2a3a-4e51-875d-131d764e1ed0" (UID: "fcef9b8a-2a3a-4e51-875d-131d764e1ed0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.373877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.372670    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-credential-socket" pod "98a5a772-0770-40c9-8a01-5b991fee55d4" (UID: "98a5a772-0770-40c9-8a01-5b991fee55d4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.373877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.372736    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-podinfo" pod "98a5a772-0770-40c9-8a01-5b991fee55d4" (UID: "98a5a772-0770-40c9-8a01-5b991fee55d4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:02:49.373877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.372783    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-envoy" pod "fcef9b8a-2a3a-4e51-875d-131d764e1ed0" (UID: "fcef9b8a-2a3a-4e51-875d-131d764e1ed0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.373877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.373768    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-token" pod "fcef9b8a-2a3a-4e51-875d-131d764e1ed0" (UID: "fcef9b8a-2a3a-4e51-875d-131d764e1ed0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:02:49.373877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.373829    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-podinfo" pod "fcef9b8a-2a3a-4e51-875d-131d764e1ed0" (UID: "fcef9b8a-2a3a-4e51-875d-131d764e1ed0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:02:49.373877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.373838    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-podinfo" pod "562448ab-2cf6-4fe5-b2f0-0c227a4ef70a" (UID: "562448ab-2cf6-4fe5-b2f0-0c227a4ef70a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:02:49.374825 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.374792    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-kube-api-access-f8q7b" pod "562448ab-2cf6-4fe5-b2f0-0c227a4ef70a" (UID: "562448ab-2cf6-4fe5-b2f0-0c227a4ef70a"). InnerVolumeSpecName "kube-api-access-f8q7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:02:49.375159 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.375088    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-envoy" pod "98a5a772-0770-40c9-8a01-5b991fee55d4" (UID: "98a5a772-0770-40c9-8a01-5b991fee55d4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.375159 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.375121    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-kube-api-access-gljr4" pod "fcef9b8a-2a3a-4e51-875d-131d764e1ed0" (UID: "fcef9b8a-2a3a-4e51-875d-131d764e1ed0"). InnerVolumeSpecName "kube-api-access-gljr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:02:49.375388 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.375362    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-data" pod "562448ab-2cf6-4fe5-b2f0-0c227a4ef70a" (UID: "562448ab-2cf6-4fe5-b2f0-0c227a4ef70a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.375485 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.375467    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-token" pod "98a5a772-0770-40c9-8a01-5b991fee55d4" (UID: "98a5a772-0770-40c9-8a01-5b991fee55d4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:02:49.376120 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.376101    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-data" pod "98a5a772-0770-40c9-8a01-5b991fee55d4" (UID: "98a5a772-0770-40c9-8a01-5b991fee55d4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.376422 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.376402    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-token" pod "562448ab-2cf6-4fe5-b2f0-0c227a4ef70a" (UID: "562448ab-2cf6-4fe5-b2f0-0c227a4ef70a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:02:49.377759 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.377742    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98a5a772-0770-40c9-8a01-5b991fee55d4-kube-api-access-ffn62" pod "98a5a772-0770-40c9-8a01-5b991fee55d4" (UID: "98a5a772-0770-40c9-8a01-5b991fee55d4"). InnerVolumeSpecName "kube-api-access-ffn62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:02:49.472362 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.472327    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffn62\" (UniqueName: \"kubernetes.io/projected/98a5a772-0770-40c9-8a01-5b991fee55d4-kube-api-access-ffn62\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.479068 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479038    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.479193 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479077    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.479193 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479101    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.479193 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479115    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.479193 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479128    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.479193 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479143    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8q7b\" (UniqueName: \"kubernetes.io/projected/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-kube-api-access-f8q7b\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.479193 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479158    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.479193 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479171    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.479193 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479184    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gljr4\" (UniqueName: \"kubernetes.io/projected/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-kube-api-access-gljr4\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.480068 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479197    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.480068 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479211    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98a5a772-0770-40c9-8a01-5b991fee55d4-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.480068 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479225    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.480068 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479238    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.480068 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479251    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.480068 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479264    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.480068 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479278    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.480068 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479292    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.480068 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479305    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.480068 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479318    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.480068 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479331    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.480068 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479343    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.480068 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479357    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcef9b8a-2a3a-4e51-875d-131d764e1ed0-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.480068 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.479370    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98a5a772-0770-40c9-8a01-5b991fee55d4-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.798873 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.798827    5820 generic.go:356] "Generic (PLEG): container finished" podID="fcef9b8a-2a3a-4e51-875d-131d764e1ed0" containerID="751f68cb8d9b7adf06705351fa4d4a2eaaaaac525c08300a50bf84c93f0b0076" exitCode=0
Oct 30 19:02:49.799237 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.799195    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"
Oct 30 19:02:49.801078 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.801047    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms" event={"ID":"fcef9b8a-2a3a-4e51-875d-131d764e1ed0","Type":"ContainerDied","Data":"751f68cb8d9b7adf06705351fa4d4a2eaaaaac525c08300a50bf84c93f0b0076"}
Oct 30 19:02:49.801313 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.801289    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms" event={"ID":"fcef9b8a-2a3a-4e51-875d-131d764e1ed0","Type":"ContainerDied","Data":"65fd26b72935535f0ebcd91a2085b7a7f9a6871f30ab7da39b2a07c2fd64ae5c"}
Oct 30 19:02:49.801509 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.801444    5820 scope.go:122] "RemoveContainer" containerID="751f68cb8d9b7adf06705351fa4d4a2eaaaaac525c08300a50bf84c93f0b0076"
Oct 30 19:02:49.810314 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.810285    5820 generic.go:356] "Generic (PLEG): container finished" podID="98a5a772-0770-40c9-8a01-5b991fee55d4" containerID="8ed8e4f60fd86b64e0d314cb04a00444d1256f43c49341a643981caf79ccee67" exitCode=0
Oct 30 19:02:49.810427 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.810340    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"
Oct 30 19:02:49.810427 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.810413    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv" event={"ID":"98a5a772-0770-40c9-8a01-5b991fee55d4","Type":"ContainerDied","Data":"8ed8e4f60fd86b64e0d314cb04a00444d1256f43c49341a643981caf79ccee67"}
Oct 30 19:02:49.810507 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.810460    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv" event={"ID":"98a5a772-0770-40c9-8a01-5b991fee55d4","Type":"ContainerDied","Data":"a79d0c8cc97964e3984efb8016a489eb40927000e5b68ea9b155e856dde52960"}
Oct 30 19:02:49.812096 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.812074    5820 generic.go:356] "Generic (PLEG): container finished" podID="562448ab-2cf6-4fe5-b2f0-0c227a4ef70a" containerID="dc2babf0a5be41fcba7e033b1f2208215296bf988cb598b0ca9ff6fa72579e1a" exitCode=0
Oct 30 19:02:49.812224 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.812122    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"
Oct 30 19:02:49.812224 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.812150    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8" event={"ID":"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a","Type":"ContainerDied","Data":"dc2babf0a5be41fcba7e033b1f2208215296bf988cb598b0ca9ff6fa72579e1a"}
Oct 30 19:02:49.812224 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.812191    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8" event={"ID":"562448ab-2cf6-4fe5-b2f0-0c227a4ef70a","Type":"ContainerDied","Data":"6294b73cc2dc8f2bc4c7ebf29de61536229c87e80c4bd74fa4b909b883c4e39c"}
Oct 30 19:02:49.813986 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.813932    5820 generic.go:356] "Generic (PLEG): container finished" podID="4be0753d-602a-4f4a-a02d-5d6baf686df2" containerID="157cc81bc9ef32c0907cd568640071754e0533cf660713eeeca407afa24f63da" exitCode=0
Oct 30 19:02:49.813986 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.813971    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6" event={"ID":"4be0753d-602a-4f4a-a02d-5d6baf686df2","Type":"ContainerDied","Data":"157cc81bc9ef32c0907cd568640071754e0533cf660713eeeca407afa24f63da"}
Oct 30 19:02:49.814123 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.814002    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"
Oct 30 19:02:49.814123 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.814016    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6" event={"ID":"4be0753d-602a-4f4a-a02d-5d6baf686df2","Type":"ContainerDied","Data":"eeecf01a8c36e917c8ff8037d0a5ff95c26d42ef89ad13de02e361f554beeaa5"}
Oct 30 19:02:49.846479 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.846459    5820 scope.go:122] "RemoveContainer" containerID="751f68cb8d9b7adf06705351fa4d4a2eaaaaac525c08300a50bf84c93f0b0076"
Oct 30 19:02:49.846776 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:02:49.846754    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"751f68cb8d9b7adf06705351fa4d4a2eaaaaac525c08300a50bf84c93f0b0076\": container with ID starting with 751f68cb8d9b7adf06705351fa4d4a2eaaaaac525c08300a50bf84c93f0b0076 not found: ID does not exist" containerID="751f68cb8d9b7adf06705351fa4d4a2eaaaaac525c08300a50bf84c93f0b0076"
Oct 30 19:02:49.846834 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.846786    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"751f68cb8d9b7adf06705351fa4d4a2eaaaaac525c08300a50bf84c93f0b0076"} err="failed to get container status \"751f68cb8d9b7adf06705351fa4d4a2eaaaaac525c08300a50bf84c93f0b0076\": rpc error: code = NotFound desc = could not find container \"751f68cb8d9b7adf06705351fa4d4a2eaaaaac525c08300a50bf84c93f0b0076\": container with ID starting with 751f68cb8d9b7adf06705351fa4d4a2eaaaaac525c08300a50bf84c93f0b0076 not found: ID does not exist"
Oct 30 19:02:49.846834 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.846808    5820 scope.go:122] "RemoveContainer" containerID="8ed8e4f60fd86b64e0d314cb04a00444d1256f43c49341a643981caf79ccee67"
Oct 30 19:02:49.857110 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.857088    5820 scope.go:122] "RemoveContainer" containerID="8ed8e4f60fd86b64e0d314cb04a00444d1256f43c49341a643981caf79ccee67"
Oct 30 19:02:49.857379 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:02:49.857354    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ed8e4f60fd86b64e0d314cb04a00444d1256f43c49341a643981caf79ccee67\": container with ID starting with 8ed8e4f60fd86b64e0d314cb04a00444d1256f43c49341a643981caf79ccee67 not found: ID does not exist" containerID="8ed8e4f60fd86b64e0d314cb04a00444d1256f43c49341a643981caf79ccee67"
Oct 30 19:02:49.857432 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.857391    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ed8e4f60fd86b64e0d314cb04a00444d1256f43c49341a643981caf79ccee67"} err="failed to get container status \"8ed8e4f60fd86b64e0d314cb04a00444d1256f43c49341a643981caf79ccee67\": rpc error: code = NotFound desc = could not find container \"8ed8e4f60fd86b64e0d314cb04a00444d1256f43c49341a643981caf79ccee67\": container with ID starting with 8ed8e4f60fd86b64e0d314cb04a00444d1256f43c49341a643981caf79ccee67 not found: ID does not exist"
Oct 30 19:02:49.857432 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.857413    5820 scope.go:122] "RemoveContainer" containerID="dc2babf0a5be41fcba7e033b1f2208215296bf988cb598b0ca9ff6fa72579e1a"
Oct 30 19:02:49.867127 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.867110    5820 scope.go:122] "RemoveContainer" containerID="dc2babf0a5be41fcba7e033b1f2208215296bf988cb598b0ca9ff6fa72579e1a"
Oct 30 19:02:49.867390 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:02:49.867361    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc2babf0a5be41fcba7e033b1f2208215296bf988cb598b0ca9ff6fa72579e1a\": container with ID starting with dc2babf0a5be41fcba7e033b1f2208215296bf988cb598b0ca9ff6fa72579e1a not found: ID does not exist" containerID="dc2babf0a5be41fcba7e033b1f2208215296bf988cb598b0ca9ff6fa72579e1a"
Oct 30 19:02:49.867504 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.867395    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc2babf0a5be41fcba7e033b1f2208215296bf988cb598b0ca9ff6fa72579e1a"} err="failed to get container status \"dc2babf0a5be41fcba7e033b1f2208215296bf988cb598b0ca9ff6fa72579e1a\": rpc error: code = NotFound desc = could not find container \"dc2babf0a5be41fcba7e033b1f2208215296bf988cb598b0ca9ff6fa72579e1a\": container with ID starting with dc2babf0a5be41fcba7e033b1f2208215296bf988cb598b0ca9ff6fa72579e1a not found: ID does not exist"
Oct 30 19:02:49.867504 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.867418    5820 scope.go:122] "RemoveContainer" containerID="157cc81bc9ef32c0907cd568640071754e0533cf660713eeeca407afa24f63da"
Oct 30 19:02:49.880724 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.880705    5820 scope.go:122] "RemoveContainer" containerID="fb8d8751512c45691d330f0e75dd6a4ab125a2bf91b0f276cfcae2350c47c972"
Oct 30 19:02:49.892569 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.892547    5820 scope.go:122] "RemoveContainer" containerID="157cc81bc9ef32c0907cd568640071754e0533cf660713eeeca407afa24f63da"
Oct 30 19:02:49.892821 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:02:49.892803    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"157cc81bc9ef32c0907cd568640071754e0533cf660713eeeca407afa24f63da\": container with ID starting with 157cc81bc9ef32c0907cd568640071754e0533cf660713eeeca407afa24f63da not found: ID does not exist" containerID="157cc81bc9ef32c0907cd568640071754e0533cf660713eeeca407afa24f63da"
Oct 30 19:02:49.892876 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.892831    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"157cc81bc9ef32c0907cd568640071754e0533cf660713eeeca407afa24f63da"} err="failed to get container status \"157cc81bc9ef32c0907cd568640071754e0533cf660713eeeca407afa24f63da\": rpc error: code = NotFound desc = could not find container \"157cc81bc9ef32c0907cd568640071754e0533cf660713eeeca407afa24f63da\": container with ID starting with 157cc81bc9ef32c0907cd568640071754e0533cf660713eeeca407afa24f63da not found: ID does not exist"
Oct 30 19:02:49.892876 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.892852    5820 scope.go:122] "RemoveContainer" containerID="fb8d8751512c45691d330f0e75dd6a4ab125a2bf91b0f276cfcae2350c47c972"
Oct 30 19:02:49.893165 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:02:49.893143    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb8d8751512c45691d330f0e75dd6a4ab125a2bf91b0f276cfcae2350c47c972\": container with ID starting with fb8d8751512c45691d330f0e75dd6a4ab125a2bf91b0f276cfcae2350c47c972 not found: ID does not exist" containerID="fb8d8751512c45691d330f0e75dd6a4ab125a2bf91b0f276cfcae2350c47c972"
Oct 30 19:02:49.893222 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.893173    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb8d8751512c45691d330f0e75dd6a4ab125a2bf91b0f276cfcae2350c47c972"} err="failed to get container status \"fb8d8751512c45691d330f0e75dd6a4ab125a2bf91b0f276cfcae2350c47c972\": rpc error: code = NotFound desc = could not find container \"fb8d8751512c45691d330f0e75dd6a4ab125a2bf91b0f276cfcae2350c47c972\": container with ID starting with fb8d8751512c45691d330f0e75dd6a4ab125a2bf91b0f276cfcae2350c47c972 not found: ID does not exist"
Oct 30 19:02:49.971072 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.971037    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"]
Oct 30 19:02:49.974420 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:49.974390    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-bnrv6"]
Oct 30 19:02:50.082400 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:50.082317    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"]
Oct 30 19:02:50.090045 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:50.090018    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-sfdc8"]
Oct 30 19:02:50.200532 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:50.200492    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"]
Oct 30 19:02:50.204296 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:50.204270    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-zj6ms"]
Oct 30 19:02:50.313004 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:50.312969    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"]
Oct 30 19:02:50.318109 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:50.318083    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-ffpcv"]
Oct 30 19:02:51.437849 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:51.437804    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4be0753d-602a-4f4a-a02d-5d6baf686df2" path="/var/lib/kubelet/pods/4be0753d-602a-4f4a-a02d-5d6baf686df2/volumes"
Oct 30 19:02:51.438530 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:51.438507    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="562448ab-2cf6-4fe5-b2f0-0c227a4ef70a" path="/var/lib/kubelet/pods/562448ab-2cf6-4fe5-b2f0-0c227a4ef70a/volumes"
Oct 30 19:02:51.439066 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:51.439045    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="98a5a772-0770-40c9-8a01-5b991fee55d4" path="/var/lib/kubelet/pods/98a5a772-0770-40c9-8a01-5b991fee55d4/volumes"
Oct 30 19:02:51.439868 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:51.439846    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fcef9b8a-2a3a-4e51-875d-131d764e1ed0" path="/var/lib/kubelet/pods/fcef9b8a-2a3a-4e51-875d-131d764e1ed0/volumes"
Oct 30 19:02:51.809488 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:51.809453    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv" podUID="ee26eec2-d615-487b-b1b4-7ef2a0cc0516" containerName="payload-processing" containerID="cri-o://1d837f42f0600466f019399526c4a7b4ff9893a680e6c83eb7fce6e02ae0a1b8" gracePeriod=25
Oct 30 19:02:52.086095 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:52.086071    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv"
Oct 30 19:02:52.124619 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:52.124592    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee26eec2-d615-487b-b1b4-7ef2a0cc0516-kube-api-access-8vhwl\" (UniqueName: \"kubernetes.io/projected/ee26eec2-d615-487b-b1b4-7ef2a0cc0516-kube-api-access-8vhwl\") pod \"ee26eec2-d615-487b-b1b4-7ef2a0cc0516\" (UID: \"ee26eec2-d615-487b-b1b4-7ef2a0cc0516\") "
Oct 30 19:02:52.124763 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:52.124666    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee26eec2-d615-487b-b1b4-7ef2a0cc0516-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee26eec2-d615-487b-b1b4-7ef2a0cc0516-plugins-config-volume\") pod \"ee26eec2-d615-487b-b1b4-7ef2a0cc0516\" (UID: \"ee26eec2-d615-487b-b1b4-7ef2a0cc0516\") "
Oct 30 19:02:52.125035 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:52.125012    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee26eec2-d615-487b-b1b4-7ef2a0cc0516-plugins-config-volume" pod "ee26eec2-d615-487b-b1b4-7ef2a0cc0516" (UID: "ee26eec2-d615-487b-b1b4-7ef2a0cc0516"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:02:52.125192 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:52.125178    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee26eec2-d615-487b-b1b4-7ef2a0cc0516-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:52.126924 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:52.126903    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee26eec2-d615-487b-b1b4-7ef2a0cc0516-kube-api-access-8vhwl" pod "ee26eec2-d615-487b-b1b4-7ef2a0cc0516" (UID: "ee26eec2-d615-487b-b1b4-7ef2a0cc0516"). InnerVolumeSpecName "kube-api-access-8vhwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:02:52.221974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:52.213767    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:52.232013 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:52.230407    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vhwl\" (UniqueName: \"kubernetes.io/projected/ee26eec2-d615-487b-b1b4-7ef2a0cc0516-kube-api-access-8vhwl\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:52.842255 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:52.842221    5820 generic.go:356] "Generic (PLEG): container finished" podID="ee26eec2-d615-487b-b1b4-7ef2a0cc0516" containerID="1d837f42f0600466f019399526c4a7b4ff9893a680e6c83eb7fce6e02ae0a1b8" exitCode=0
Oct 30 19:02:52.842698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:52.842262    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv" event={"ID":"ee26eec2-d615-487b-b1b4-7ef2a0cc0516","Type":"ContainerDied","Data":"1d837f42f0600466f019399526c4a7b4ff9893a680e6c83eb7fce6e02ae0a1b8"}
Oct 30 19:02:52.842698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:52.842306    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv"
Oct 30 19:02:52.842698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:52.842355    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv" event={"ID":"ee26eec2-d615-487b-b1b4-7ef2a0cc0516","Type":"ContainerDied","Data":"d4edca78e268801f5b083c61a4b8ca1dc63ee892a2e559d4e2197d571e3aad51"}
Oct 30 19:02:52.842698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:52.842386    5820 scope.go:122] "RemoveContainer" containerID="1d837f42f0600466f019399526c4a7b4ff9893a680e6c83eb7fce6e02ae0a1b8"
Oct 30 19:02:52.852831 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:52.852812    5820 scope.go:122] "RemoveContainer" containerID="1d837f42f0600466f019399526c4a7b4ff9893a680e6c83eb7fce6e02ae0a1b8"
Oct 30 19:02:52.853145 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:02:52.853125    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d837f42f0600466f019399526c4a7b4ff9893a680e6c83eb7fce6e02ae0a1b8\": container with ID starting with 1d837f42f0600466f019399526c4a7b4ff9893a680e6c83eb7fce6e02ae0a1b8 not found: ID does not exist" containerID="1d837f42f0600466f019399526c4a7b4ff9893a680e6c83eb7fce6e02ae0a1b8"
Oct 30 19:02:52.853199 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:52.853154    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d837f42f0600466f019399526c4a7b4ff9893a680e6c83eb7fce6e02ae0a1b8"} err="failed to get container status \"1d837f42f0600466f019399526c4a7b4ff9893a680e6c83eb7fce6e02ae0a1b8\": rpc error: code = NotFound desc = could not find container \"1d837f42f0600466f019399526c4a7b4ff9893a680e6c83eb7fce6e02ae0a1b8\": container with ID starting with 1d837f42f0600466f019399526c4a7b4ff9893a680e6c83eb7fce6e02ae0a1b8 not found: ID does not exist"
Oct 30 19:02:53.007940 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:53.007910    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv"]
Oct 30 19:02:53.057178 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:53.057150    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-132af1-787d75b88d-pp7pv"]
Oct 30 19:02:53.438368 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:53.438336    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee26eec2-d615-487b-b1b4-7ef2a0cc0516" path="/var/lib/kubelet/pods/ee26eec2-d615-487b-b1b4-7ef2a0cc0516/volumes"
Oct 30 19:02:55.903561 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:55.903524    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:56.985383 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:02:56.985347    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:01.962409 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:01.959613    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:06.830975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:06.827994    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:07.663991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:07.663724    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:08.203371 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:03:08.203336    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:03:09.690393 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:09.690349    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:10.891975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:10.883866    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"]
Oct 30 19:03:10.891975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:10.884172    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf" podUID="bc05bef2-24fa-4808-a429-8c6d450ed396" containerName="maas-api" containerID="cri-o://20e4b7e680b1efa699d373484487aeacf67cd9586249171c0060a18e580dba5f" gracePeriod=30
Oct 30 19:03:11.035236 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:11.031177    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:11.051908 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:11.051877    5820 generic.go:356] "Generic (PLEG): container finished" podID="bc05bef2-24fa-4808-a429-8c6d450ed396" containerID="20e4b7e680b1efa699d373484487aeacf67cd9586249171c0060a18e580dba5f" exitCode=0
Oct 30 19:03:11.052051 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:11.052003    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf" event={"ID":"bc05bef2-24fa-4808-a429-8c6d450ed396","Type":"ContainerDied","Data":"20e4b7e680b1efa699d373484487aeacf67cd9586249171c0060a18e580dba5f"}
Oct 30 19:03:11.285319 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:11.285297    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"
Oct 30 19:03:11.427471 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:11.427435    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bc05bef2-24fa-4808-a429-8c6d450ed396-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc05bef2-24fa-4808-a429-8c6d450ed396-maas-api-tls\") pod \"bc05bef2-24fa-4808-a429-8c6d450ed396\" (UID: \"bc05bef2-24fa-4808-a429-8c6d450ed396\") "
Oct 30 19:03:11.427747 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:11.427724    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc05bef2-24fa-4808-a429-8c6d450ed396-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc05bef2-24fa-4808-a429-8c6d450ed396-openshift-service-ca\") pod \"bc05bef2-24fa-4808-a429-8c6d450ed396\" (UID: \"bc05bef2-24fa-4808-a429-8c6d450ed396\") "
Oct 30 19:03:11.428029 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:11.428011    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bc05bef2-24fa-4808-a429-8c6d450ed396-metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc05bef2-24fa-4808-a429-8c6d450ed396-metrics-certs\") pod \"bc05bef2-24fa-4808-a429-8c6d450ed396\" (UID: \"bc05bef2-24fa-4808-a429-8c6d450ed396\") "
Oct 30 19:03:11.428194 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:11.428180    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc05bef2-24fa-4808-a429-8c6d450ed396-kube-api-access-6ftd7\" (UniqueName: \"kubernetes.io/projected/bc05bef2-24fa-4808-a429-8c6d450ed396-kube-api-access-6ftd7\") pod \"bc05bef2-24fa-4808-a429-8c6d450ed396\" (UID: \"bc05bef2-24fa-4808-a429-8c6d450ed396\") "
Oct 30 19:03:11.433036 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:11.431102    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc05bef2-24fa-4808-a429-8c6d450ed396-openshift-service-ca" pod "bc05bef2-24fa-4808-a429-8c6d450ed396" (UID: "bc05bef2-24fa-4808-a429-8c6d450ed396"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:03:11.440442 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:11.440365    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc05bef2-24fa-4808-a429-8c6d450ed396-maas-api-tls" pod "bc05bef2-24fa-4808-a429-8c6d450ed396" (UID: "bc05bef2-24fa-4808-a429-8c6d450ed396"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:03:11.446991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:11.442217    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc05bef2-24fa-4808-a429-8c6d450ed396-metrics-certs" pod "bc05bef2-24fa-4808-a429-8c6d450ed396" (UID: "bc05bef2-24fa-4808-a429-8c6d450ed396"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:03:11.449160 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:11.449134    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc05bef2-24fa-4808-a429-8c6d450ed396-kube-api-access-6ftd7" pod "bc05bef2-24fa-4808-a429-8c6d450ed396" (UID: "bc05bef2-24fa-4808-a429-8c6d450ed396"). InnerVolumeSpecName "kube-api-access-6ftd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:03:11.534302 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:11.534268    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc05bef2-24fa-4808-a429-8c6d450ed396-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:11.534302 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:11.534301    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc05bef2-24fa-4808-a429-8c6d450ed396-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:11.534509 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:11.534313    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc05bef2-24fa-4808-a429-8c6d450ed396-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:11.534509 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:11.534324    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ftd7\" (UniqueName: \"kubernetes.io/projected/bc05bef2-24fa-4808-a429-8c6d450ed396-kube-api-access-6ftd7\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:12.060149 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:12.060118    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"
Oct 30 19:03:12.060149 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:12.060137    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf" event={"ID":"bc05bef2-24fa-4808-a429-8c6d450ed396","Type":"ContainerDied","Data":"e5d91cc41fa99ccd658e57767fcc3732e06fb0199792ac66ea49309142a8a0f9"}
Oct 30 19:03:12.060660 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:12.060198    5820 scope.go:122] "RemoveContainer" containerID="20e4b7e680b1efa699d373484487aeacf67cd9586249171c0060a18e580dba5f"
Oct 30 19:03:12.225005 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:12.224973    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"]
Oct 30 19:03:12.228532 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:12.228484    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:12.229268 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:12.229249    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-470093-799d857fdc-fckcf"]
Oct 30 19:03:12.580555 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:12.580496    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:12.869757 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:12.869666    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:13.437492 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:13.437462    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bc05bef2-24fa-4808-a429-8c6d450ed396" path="/var/lib/kubelet/pods/bc05bef2-24fa-4808-a429-8c6d450ed396/volumes"
Oct 30 19:03:13.582830 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:13.582794    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:13.852176 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:13.852143    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:14.298667 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:14.291780    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:14.472338 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:14.472298    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:14.937441 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:14.937413    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:16.208022 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:16.207928    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:16.473655 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:16.473576    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:16.569040 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:16.569011    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"]
Oct 30 19:03:16.569278 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:16.569253    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6" podUID="1aac21cb-584a-42ad-9dc6-99aada4cdba1" containerName="istio-proxy" containerID="cri-o://d751a058232056c53b8098256b80e74d9a55701ec0380e5b40332de1d1a78149" gracePeriod=30
Oct 30 19:03:16.581376 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:16.581352    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"]
Oct 30 19:03:16.581586 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:16.581560    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg" podUID="e4ccd454-b219-4d0c-af05-289d90cfe455" containerName="istio-proxy" containerID="cri-o://694e1f97c38f20709977d2ba4b28ecb91eee74347f4cc0cbf2c77f8261c5cb07" gracePeriod=30
Oct 30 19:03:16.594373 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:16.594349    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"]
Oct 30 19:03:16.594559 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:16.594539    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b" podUID="370e902d-498f-454d-bacc-1edaf55bcd76" containerName="istio-proxy" containerID="cri-o://83d1b426c700e2a25af84cd3101733b8de805c951f295819d6a831f9e7d663c8" gracePeriod=30
Oct 30 19:03:16.721976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:16.721156    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:17.186509 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.186474    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn"]
Oct 30 19:03:17.297852 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.297720    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr"]
Oct 30 19:03:17.410219 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.410190    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:17.662694 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.662034    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:17.842880 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.842843    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:17.893593 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.893564    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"]
Oct 30 19:03:17.894421 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.894399    5820 cpu_manager.go:414] "containerMap: removing container" podUID="4be0753d-602a-4f4a-a02d-5d6baf686df2" containerName="istio-proxy"
Oct 30 19:03:17.894421 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.894418    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4be0753d-602a-4f4a-a02d-5d6baf686df2" containerName="istio-proxy"
Oct 30 19:03:17.894421 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.894434    5820 cpu_manager.go:414] "containerMap: removing container" podUID="d2844bd5-87c5-4f3c-86d4-d4041c484355" containerName="istio-proxy"
Oct 30 19:03:17.894620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.894441    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d2844bd5-87c5-4f3c-86d4-d4041c484355" containerName="istio-proxy"
Oct 30 19:03:17.894620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.894452    5820 cpu_manager.go:414] "containerMap: removing container" podUID="fcef9b8a-2a3a-4e51-875d-131d764e1ed0" containerName="istio-proxy"
Oct 30 19:03:17.894620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.894456    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fcef9b8a-2a3a-4e51-875d-131d764e1ed0" containerName="istio-proxy"
Oct 30 19:03:17.894620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.894477    5820 cpu_manager.go:414] "containerMap: removing container" podUID="bc05bef2-24fa-4808-a429-8c6d450ed396" containerName="maas-api"
Oct 30 19:03:17.894620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.894482    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc05bef2-24fa-4808-a429-8c6d450ed396" containerName="maas-api"
Oct 30 19:03:17.894620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.894492    5820 cpu_manager.go:414] "containerMap: removing container" podUID="ee26eec2-d615-487b-b1b4-7ef2a0cc0516" containerName="payload-processing"
Oct 30 19:03:17.894620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.894496    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee26eec2-d615-487b-b1b4-7ef2a0cc0516" containerName="payload-processing"
Oct 30 19:03:17.894620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.894503    5820 cpu_manager.go:414] "containerMap: removing container" podUID="562448ab-2cf6-4fe5-b2f0-0c227a4ef70a" containerName="istio-proxy"
Oct 30 19:03:17.894620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.894508    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="562448ab-2cf6-4fe5-b2f0-0c227a4ef70a" containerName="istio-proxy"
Oct 30 19:03:17.894620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.894534    5820 cpu_manager.go:414] "containerMap: removing container" podUID="98a5a772-0770-40c9-8a01-5b991fee55d4" containerName="istio-proxy"
Oct 30 19:03:17.894620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.894540    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="98a5a772-0770-40c9-8a01-5b991fee55d4" containerName="istio-proxy"
Oct 30 19:03:17.894620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.894624    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="98a5a772-0770-40c9-8a01-5b991fee55d4" containerName="istio-proxy"
Oct 30 19:03:17.895047 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.894632    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="d2844bd5-87c5-4f3c-86d4-d4041c484355" containerName="istio-proxy"
Oct 30 19:03:17.895047 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.894639    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="4be0753d-602a-4f4a-a02d-5d6baf686df2" containerName="istio-proxy"
Oct 30 19:03:17.895047 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.894645    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="4be0753d-602a-4f4a-a02d-5d6baf686df2" containerName="istio-proxy"
Oct 30 19:03:17.895047 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.894652    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="fcef9b8a-2a3a-4e51-875d-131d764e1ed0" containerName="istio-proxy"
Oct 30 19:03:17.895047 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.894662    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc05bef2-24fa-4808-a429-8c6d450ed396" containerName="maas-api"
Oct 30 19:03:17.895047 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.894668    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee26eec2-d615-487b-b1b4-7ef2a0cc0516" containerName="payload-processing"
Oct 30 19:03:17.895047 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.894674    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="562448ab-2cf6-4fe5-b2f0-0c227a4ef70a" containerName="istio-proxy"
Oct 30 19:03:17.900294 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.900277    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:17.948983 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.948885    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-envoy\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:17.949223 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.949041    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v74b2\" (UniqueName: \"kubernetes.io/projected/18a0d7fb-d2d5-49b6-9d2c-be64503459de-kube-api-access-v74b2\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:17.949223 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.949100    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-data\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:17.949341 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.949213    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-podinfo\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:17.949341 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.949257    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-workload-socket\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:17.949469 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.949442    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-workload-certs\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:17.949614 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.949516    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istiod-ca-cert\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:17.949614 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.949567    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-token\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:17.949720 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.949647    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-credential-socket\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:17.958096 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.957631    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-842bbd-openshift-default-dockercfg-4rsqr\""
Oct 30 19:03:17.960820 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:17.960761    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"]
Oct 30 19:03:18.036308 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.036247    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:18.068521 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.056881    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-credential-socket\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:18.081003 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.069463    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-credential-socket\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:18.100282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.073137    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-envoy\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:18.100282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.092180    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v74b2\" (UniqueName: \"kubernetes.io/projected/18a0d7fb-d2d5-49b6-9d2c-be64503459de-kube-api-access-v74b2\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:18.100282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.092255    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-data\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:18.100282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.092317    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-podinfo\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:18.100282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.092341    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-workload-socket\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:18.100282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.092513    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-workload-certs\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:18.100282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.092605    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istiod-ca-cert\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:18.100282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.092665    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-token\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:18.100282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.093689    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-data\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:18.100282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.094391    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-workload-certs\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:18.100282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.094712    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-workload-socket\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:18.100282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.095368    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istiod-ca-cert\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:18.112229 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.112152    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-envoy\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:18.116458 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.116391    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-podinfo\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:18.164781 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.164554    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-token\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:18.169846 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.169811    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v74b2\" (UniqueName: \"kubernetes.io/projected/18a0d7fb-d2d5-49b6-9d2c-be64503459de-kube-api-access-v74b2\") pod \"e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") " pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:18.220026 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.214386    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:18.302034 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.301838    5820 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:03:18.302034 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.301897    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6" podUID="1aac21cb-584a-42ad-9dc6-99aada4cdba1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:03:18.387679 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.387643    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:18.687746 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.687720    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"]
Oct 30 19:03:18.689664 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:03:18.689636    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18a0d7fb_d2d5_49b6_9d2c_be64503459de.slice/crio-84dd47570ca8d884adfb25f29cf789827a4d53f0077793b0070646f192ec1028 WatchSource:0}: Error finding container 84dd47570ca8d884adfb25f29cf789827a4d53f0077793b0070646f192ec1028: Status 404 returned error can't find the container with id 84dd47570ca8d884adfb25f29cf789827a4d53f0077793b0070646f192ec1028
Oct 30 19:03:18.692186 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.692153    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:03:18.692270 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.692219    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:03:18.692270 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:18.692246    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:03:19.112589 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:19.112555    5820 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:03:19.113291 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:19.113255    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg" podUID="e4ccd454-b219-4d0c-af05-289d90cfe455" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:03:19.147989 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:19.147198    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8" event={"ID":"18a0d7fb-d2d5-49b6-9d2c-be64503459de","Type":"ContainerStarted","Data":"d8b91a1a2211449bc4a71de67b331434aec2c04a4e34f8a39fbb58d40438cd62"}
Oct 30 19:03:19.147989 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:19.147267    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8" event={"ID":"18a0d7fb-d2d5-49b6-9d2c-be64503459de","Type":"ContainerStarted","Data":"84dd47570ca8d884adfb25f29cf789827a4d53f0077793b0070646f192ec1028"}
Oct 30 19:03:19.167365 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:19.167330    5820 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:03:19.167678 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:19.167621    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b" podUID="370e902d-498f-454d-bacc-1edaf55bcd76" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:03:19.216246 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:19.216215    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:19.217824 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:19.217799    5820 patch_prober.go:29] interesting pod/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 19:03:19.217968 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:19.217848    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8" podUID="18a0d7fb-d2d5-49b6-9d2c-be64503459de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 19:03:19.694977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:19.693007    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:20.220872 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:20.220842    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:20.350789 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:20.350741    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8" podStartSLOduration=3.350725084 podStartE2EDuration="3.350725084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:03:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:03:19.283442721 +0000 UTC m=+1719.995191422" watchObservedRunningTime="2026-10-30 19:03:20.350725084 +0000 UTC m=+1721.062473780"
Oct 30 19:03:21.053972 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:21.041914    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:21.159313 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:21.159282    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:21.160107 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:21.160088    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:03:21.231608 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:21.231572    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:21.474958 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:21.474868    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:21.894188 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:21.894155    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:03:21.944588 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:21.944563    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:03:21.948098 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:21.948080    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:03:22.006347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.006278    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/370e902d-498f-454d-bacc-1edaf55bcd76-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/370e902d-498f-454d-bacc-1edaf55bcd76-istiod-ca-cert\") pod \"370e902d-498f-454d-bacc-1edaf55bcd76\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") "
Oct 30 19:03:22.006494 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.006358    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-istio-envoy\") pod \"370e902d-498f-454d-bacc-1edaf55bcd76\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") "
Oct 30 19:03:22.006494 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.006445    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-credential-socket\") pod \"e4ccd454-b219-4d0c-af05-289d90cfe455\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") "
Oct 30 19:03:22.006618 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.006496    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e4ccd454-b219-4d0c-af05-289d90cfe455-kube-api-access-hlp2t\" (UniqueName: \"kubernetes.io/projected/e4ccd454-b219-4d0c-af05-289d90cfe455-kube-api-access-hlp2t\") pod \"e4ccd454-b219-4d0c-af05-289d90cfe455\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") "
Oct 30 19:03:22.006618 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.006556    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-envoy\") pod \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") "
Oct 30 19:03:22.006618 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.006589    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/370e902d-498f-454d-bacc-1edaf55bcd76-kube-api-access-9ldhq\" (UniqueName: \"kubernetes.io/projected/370e902d-498f-454d-bacc-1edaf55bcd76-kube-api-access-9ldhq\") pod \"370e902d-498f-454d-bacc-1edaf55bcd76\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") "
Oct 30 19:03:22.006780 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.006619    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-credential-socket\") pod \"370e902d-498f-454d-bacc-1edaf55bcd76\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") "
Oct 30 19:03:22.006780 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.006656    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-workload-socket\") pod \"e4ccd454-b219-4d0c-af05-289d90cfe455\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") "
Oct 30 19:03:22.006780 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.006710    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istiod-ca-cert\") pod \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") "
Oct 30 19:03:22.006780 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.006722    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-credential-socket" pod "e4ccd454-b219-4d0c-af05-289d90cfe455" (UID: "e4ccd454-b219-4d0c-af05-289d90cfe455"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.006780 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.006737    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1aac21cb-584a-42ad-9dc6-99aada4cdba1-kube-api-access-c4zhg\" (UniqueName: \"kubernetes.io/projected/1aac21cb-584a-42ad-9dc6-99aada4cdba1-kube-api-access-c4zhg\") pod \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") "
Oct 30 19:03:22.006780 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.006745    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/370e902d-498f-454d-bacc-1edaf55bcd76-istiod-ca-cert" pod "370e902d-498f-454d-bacc-1edaf55bcd76" (UID: "370e902d-498f-454d-bacc-1edaf55bcd76"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:03:22.007144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.006797    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-workload-socket\") pod \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") "
Oct 30 19:03:22.007144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.006839    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/370e902d-498f-454d-bacc-1edaf55bcd76-istio-token\" (UniqueName: \"kubernetes.io/projected/370e902d-498f-454d-bacc-1edaf55bcd76-istio-token\") pod \"370e902d-498f-454d-bacc-1edaf55bcd76\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") "
Oct 30 19:03:22.007144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.006890    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-credential-socket\") pod \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") "
Oct 30 19:03:22.007144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.007033    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/370e902d-498f-454d-bacc-1edaf55bcd76-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/370e902d-498f-454d-bacc-1edaf55bcd76-istio-podinfo\") pod \"370e902d-498f-454d-bacc-1edaf55bcd76\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") "
Oct 30 19:03:22.007144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.007067    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-workload-socket" pod "e4ccd454-b219-4d0c-af05-289d90cfe455" (UID: "e4ccd454-b219-4d0c-af05-289d90cfe455"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.007144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.007069    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-workload-certs\") pod \"e4ccd454-b219-4d0c-af05-289d90cfe455\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") "
Oct 30 19:03:22.007144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.007143    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-podinfo\") pod \"e4ccd454-b219-4d0c-af05-289d90cfe455\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") "
Oct 30 19:03:22.007471 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.007181    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-workload-certs\") pod \"370e902d-498f-454d-bacc-1edaf55bcd76\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") "
Oct 30 19:03:22.007471 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.007250    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-podinfo\") pod \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") "
Oct 30 19:03:22.007471 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.007285    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-data\") pod \"e4ccd454-b219-4d0c-af05-289d90cfe455\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") "
Oct 30 19:03:22.007471 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.007323    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-workload-certs" pod "e4ccd454-b219-4d0c-af05-289d90cfe455" (UID: "e4ccd454-b219-4d0c-af05-289d90cfe455"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.007471 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.007330    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-istio-data\") pod \"370e902d-498f-454d-bacc-1edaf55bcd76\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") "
Oct 30 19:03:22.007471 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.007362    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-credential-socket" pod "370e902d-498f-454d-bacc-1edaf55bcd76" (UID: "370e902d-498f-454d-bacc-1edaf55bcd76"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.007471 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.007436    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-envoy\") pod \"e4ccd454-b219-4d0c-af05-289d90cfe455\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") "
Oct 30 19:03:22.007824 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.007479    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-data\") pod \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") "
Oct 30 19:03:22.007824 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.007506    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-token\" (UniqueName: \"kubernetes.io/projected/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-token\") pod \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") "
Oct 30 19:03:22.007824 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.007553    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-workload-certs\") pod \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\" (UID: \"1aac21cb-584a-42ad-9dc6-99aada4cdba1\") "
Oct 30 19:03:22.007824 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.007595    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-token\" (UniqueName: \"kubernetes.io/projected/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-token\") pod \"e4ccd454-b219-4d0c-af05-289d90cfe455\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") "
Oct 30 19:03:22.007824 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.007643    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e4ccd454-b219-4d0c-af05-289d90cfe455-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4ccd454-b219-4d0c-af05-289d90cfe455-istiod-ca-cert\") pod \"e4ccd454-b219-4d0c-af05-289d90cfe455\" (UID: \"e4ccd454-b219-4d0c-af05-289d90cfe455\") "
Oct 30 19:03:22.007824 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.007699    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-workload-socket\") pod \"370e902d-498f-454d-bacc-1edaf55bcd76\" (UID: \"370e902d-498f-454d-bacc-1edaf55bcd76\") "
Oct 30 19:03:22.007824 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.007716    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-data" pod "e4ccd454-b219-4d0c-af05-289d90cfe455" (UID: "e4ccd454-b219-4d0c-af05-289d90cfe455"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.008880 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.008123    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-workload-certs" pod "370e902d-498f-454d-bacc-1edaf55bcd76" (UID: "370e902d-498f-454d-bacc-1edaf55bcd76"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.008880 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.008321    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-workload-socket" pod "370e902d-498f-454d-bacc-1edaf55bcd76" (UID: "370e902d-498f-454d-bacc-1edaf55bcd76"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.008880 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.008397    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istiod-ca-cert" pod "1aac21cb-584a-42ad-9dc6-99aada4cdba1" (UID: "1aac21cb-584a-42ad-9dc6-99aada4cdba1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:03:22.008880 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.008505    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-workload-socket" pod "1aac21cb-584a-42ad-9dc6-99aada4cdba1" (UID: "1aac21cb-584a-42ad-9dc6-99aada4cdba1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.008880 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.008671    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.008880 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.008685    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-credential-socket" pod "1aac21cb-584a-42ad-9dc6-99aada4cdba1" (UID: "1aac21cb-584a-42ad-9dc6-99aada4cdba1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.008880 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.008694    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.008880 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.008712    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.008880 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.008725    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/370e902d-498f-454d-bacc-1edaf55bcd76-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.008880 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.008741    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.008880 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.008755    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.008880 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.008771    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.008880 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.008785    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.008880 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.008799    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.008880 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.008814    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.009849 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.009018    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-workload-certs" pod "1aac21cb-584a-42ad-9dc6-99aada4cdba1" (UID: "1aac21cb-584a-42ad-9dc6-99aada4cdba1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.009849 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.009384    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4ccd454-b219-4d0c-af05-289d90cfe455-istiod-ca-cert" pod "e4ccd454-b219-4d0c-af05-289d90cfe455" (UID: "e4ccd454-b219-4d0c-af05-289d90cfe455"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:03:22.009849 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.009632    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-data" pod "1aac21cb-584a-42ad-9dc6-99aada4cdba1" (UID: "1aac21cb-584a-42ad-9dc6-99aada4cdba1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.009849 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.009830    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-istio-data" pod "370e902d-498f-454d-bacc-1edaf55bcd76" (UID: "370e902d-498f-454d-bacc-1edaf55bcd76"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.010445 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.010404    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aac21cb-584a-42ad-9dc6-99aada4cdba1-kube-api-access-c4zhg" pod "1aac21cb-584a-42ad-9dc6-99aada4cdba1" (UID: "1aac21cb-584a-42ad-9dc6-99aada4cdba1"). InnerVolumeSpecName "kube-api-access-c4zhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:03:22.010568 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.010541    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-istio-envoy" pod "370e902d-498f-454d-bacc-1edaf55bcd76" (UID: "370e902d-498f-454d-bacc-1edaf55bcd76"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.012273 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.012244    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/370e902d-498f-454d-bacc-1edaf55bcd76-istio-podinfo" pod "370e902d-498f-454d-bacc-1edaf55bcd76" (UID: "370e902d-498f-454d-bacc-1edaf55bcd76"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:03:22.013328 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.013303    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-envoy" pod "e4ccd454-b219-4d0c-af05-289d90cfe455" (UID: "e4ccd454-b219-4d0c-af05-289d90cfe455"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.014297 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.014275    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-podinfo" pod "e4ccd454-b219-4d0c-af05-289d90cfe455" (UID: "e4ccd454-b219-4d0c-af05-289d90cfe455"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:03:22.014591 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.014451    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-envoy" pod "1aac21cb-584a-42ad-9dc6-99aada4cdba1" (UID: "1aac21cb-584a-42ad-9dc6-99aada4cdba1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.014591 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.014468    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/370e902d-498f-454d-bacc-1edaf55bcd76-kube-api-access-9ldhq" pod "370e902d-498f-454d-bacc-1edaf55bcd76" (UID: "370e902d-498f-454d-bacc-1edaf55bcd76"). InnerVolumeSpecName "kube-api-access-9ldhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:03:22.014591 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.014486    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/370e902d-498f-454d-bacc-1edaf55bcd76-istio-token" pod "370e902d-498f-454d-bacc-1edaf55bcd76" (UID: "370e902d-498f-454d-bacc-1edaf55bcd76"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:03:22.014591 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.014524    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4ccd454-b219-4d0c-af05-289d90cfe455-kube-api-access-hlp2t" pod "e4ccd454-b219-4d0c-af05-289d90cfe455" (UID: "e4ccd454-b219-4d0c-af05-289d90cfe455"). InnerVolumeSpecName "kube-api-access-hlp2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:03:22.015020 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.014935    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-podinfo" pod "1aac21cb-584a-42ad-9dc6-99aada4cdba1" (UID: "1aac21cb-584a-42ad-9dc6-99aada4cdba1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:03:22.015125 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.015110    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-token" pod "1aac21cb-584a-42ad-9dc6-99aada4cdba1" (UID: "1aac21cb-584a-42ad-9dc6-99aada4cdba1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:03:22.015934 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.015909    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-token" pod "e4ccd454-b219-4d0c-af05-289d90cfe455" (UID: "e4ccd454-b219-4d0c-af05-289d90cfe455"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:03:22.109037 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.108845    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:22.110133 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.110109    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.110262 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.110135    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.110262 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.110149    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.110262 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.110166    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.110262 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.110178    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4ccd454-b219-4d0c-af05-289d90cfe455-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.110262 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.110190    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.110262 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.110206    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlp2t\" (UniqueName: \"kubernetes.io/projected/e4ccd454-b219-4d0c-af05-289d90cfe455-kube-api-access-hlp2t\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.110262 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.110219    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.110262 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.110231    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ldhq\" (UniqueName: \"kubernetes.io/projected/370e902d-498f-454d-bacc-1edaf55bcd76-kube-api-access-9ldhq\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.110262 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.110244    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4zhg\" (UniqueName: \"kubernetes.io/projected/1aac21cb-584a-42ad-9dc6-99aada4cdba1-kube-api-access-c4zhg\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.110262 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.110257    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/370e902d-498f-454d-bacc-1edaf55bcd76-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.115998 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.110270    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aac21cb-584a-42ad-9dc6-99aada4cdba1-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.115998 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.110283    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/370e902d-498f-454d-bacc-1edaf55bcd76-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.115998 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.110296    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.115998 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.110309    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aac21cb-584a-42ad-9dc6-99aada4cdba1-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.115998 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.110322    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/370e902d-498f-454d-bacc-1edaf55bcd76-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.115998 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.110336    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4ccd454-b219-4d0c-af05-289d90cfe455-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.185211 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.185181    5820 generic.go:356] "Generic (PLEG): container finished" podID="370e902d-498f-454d-bacc-1edaf55bcd76" containerID="83d1b426c700e2a25af84cd3101733b8de805c951f295819d6a831f9e7d663c8" exitCode=0
Oct 30 19:03:22.185378 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.185254    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"
Oct 30 19:03:22.185378 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.185266    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b" event={"ID":"370e902d-498f-454d-bacc-1edaf55bcd76","Type":"ContainerDied","Data":"83d1b426c700e2a25af84cd3101733b8de805c951f295819d6a831f9e7d663c8"}
Oct 30 19:03:22.185378 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.185359    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b" event={"ID":"370e902d-498f-454d-bacc-1edaf55bcd76","Type":"ContainerDied","Data":"02836f181faa6236826d6e029ead6e6f4ab02735488136f2cce1a31fb0c0ffec"}
Oct 30 19:03:22.185552 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.185388    5820 scope.go:122] "RemoveContainer" containerID="83d1b426c700e2a25af84cd3101733b8de805c951f295819d6a831f9e7d663c8"
Oct 30 19:03:22.186805 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.186777    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn" podUID="efd8ffd1-02a7-4096-8206-de01acd83987" containerName="payload-processing" containerID="cri-o://ff05bf57900c1530ac2401e9a92088d51e690bcda11ec35f6006af60d95cc5ce" gracePeriod=25
Oct 30 19:03:22.187195 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.187174    5820 generic.go:356] "Generic (PLEG): container finished" podID="1aac21cb-584a-42ad-9dc6-99aada4cdba1" containerID="d751a058232056c53b8098256b80e74d9a55701ec0380e5b40332de1d1a78149" exitCode=0
Oct 30 19:03:22.187479 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.187458    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6" event={"ID":"1aac21cb-584a-42ad-9dc6-99aada4cdba1","Type":"ContainerDied","Data":"d751a058232056c53b8098256b80e74d9a55701ec0380e5b40332de1d1a78149"}
Oct 30 19:03:22.187587 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.187507    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6" event={"ID":"1aac21cb-584a-42ad-9dc6-99aada4cdba1","Type":"ContainerDied","Data":"7ff789c5cf8c6294890f7fc4d505c31f2bb1a520945aaf26cdc20da738513375"}
Oct 30 19:03:22.187587 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.187584    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"
Oct 30 19:03:22.192992 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.191605    5820 generic.go:356] "Generic (PLEG): container finished" podID="e4ccd454-b219-4d0c-af05-289d90cfe455" containerID="694e1f97c38f20709977d2ba4b28ecb91eee74347f4cc0cbf2c77f8261c5cb07" exitCode=0
Oct 30 19:03:22.192992 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.191806    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"
Oct 30 19:03:22.192992 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.191921    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg" event={"ID":"e4ccd454-b219-4d0c-af05-289d90cfe455","Type":"ContainerDied","Data":"694e1f97c38f20709977d2ba4b28ecb91eee74347f4cc0cbf2c77f8261c5cb07"}
Oct 30 19:03:22.192992 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.191996    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg" event={"ID":"e4ccd454-b219-4d0c-af05-289d90cfe455","Type":"ContainerDied","Data":"f1c7cefd357ceeb3b3162f275b424d02fa82b5eb5b30e696440d0ca93c550da8"}
Oct 30 19:03:22.199861 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.199842    5820 scope.go:122] "RemoveContainer" containerID="83d1b426c700e2a25af84cd3101733b8de805c951f295819d6a831f9e7d663c8"
Oct 30 19:03:22.200932 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:03:22.200652    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83d1b426c700e2a25af84cd3101733b8de805c951f295819d6a831f9e7d663c8\": container with ID starting with 83d1b426c700e2a25af84cd3101733b8de805c951f295819d6a831f9e7d663c8 not found: ID does not exist" containerID="83d1b426c700e2a25af84cd3101733b8de805c951f295819d6a831f9e7d663c8"
Oct 30 19:03:22.200932 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.200684    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83d1b426c700e2a25af84cd3101733b8de805c951f295819d6a831f9e7d663c8"} err="failed to get container status \"83d1b426c700e2a25af84cd3101733b8de805c951f295819d6a831f9e7d663c8\": rpc error: code = NotFound desc = could not find container \"83d1b426c700e2a25af84cd3101733b8de805c951f295819d6a831f9e7d663c8\": container with ID starting with 83d1b426c700e2a25af84cd3101733b8de805c951f295819d6a831f9e7d663c8 not found: ID does not exist"
Oct 30 19:03:22.200932 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.200701    5820 scope.go:122] "RemoveContainer" containerID="d751a058232056c53b8098256b80e74d9a55701ec0380e5b40332de1d1a78149"
Oct 30 19:03:22.299032 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.299000    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr" podUID="6a9f6116-2dd0-4f64-b212-0acf561d5307" containerName="payload-pre-processing" containerID="cri-o://58c3898193a07c6134acfb024d8f0669218b3c962b3a2a3eba42b86450a31401" gracePeriod=25
Oct 30 19:03:22.304465 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.304449    5820 scope.go:122] "RemoveContainer" containerID="d751a058232056c53b8098256b80e74d9a55701ec0380e5b40332de1d1a78149"
Oct 30 19:03:22.304851 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:03:22.304829    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d751a058232056c53b8098256b80e74d9a55701ec0380e5b40332de1d1a78149\": container with ID starting with d751a058232056c53b8098256b80e74d9a55701ec0380e5b40332de1d1a78149 not found: ID does not exist" containerID="d751a058232056c53b8098256b80e74d9a55701ec0380e5b40332de1d1a78149"
Oct 30 19:03:22.304918 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.304858    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d751a058232056c53b8098256b80e74d9a55701ec0380e5b40332de1d1a78149"} err="failed to get container status \"d751a058232056c53b8098256b80e74d9a55701ec0380e5b40332de1d1a78149\": rpc error: code = NotFound desc = could not find container \"d751a058232056c53b8098256b80e74d9a55701ec0380e5b40332de1d1a78149\": container with ID starting with d751a058232056c53b8098256b80e74d9a55701ec0380e5b40332de1d1a78149 not found: ID does not exist"
Oct 30 19:03:22.304918 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.304876    5820 scope.go:122] "RemoveContainer" containerID="694e1f97c38f20709977d2ba4b28ecb91eee74347f4cc0cbf2c77f8261c5cb07"
Oct 30 19:03:22.314748 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.314723    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:22.380334 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.375910    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"]
Oct 30 19:03:22.380334 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.376191    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn" podUID="a98db9fc-3545-4eca-a2b5-964583229a01" containerName="istio-proxy" containerID="cri-o://d42168379ca9c4657fbfecb07c367460e0ccc4dc1ffcc083a40c0bc38cd86ad2" gracePeriod=30
Oct 30 19:03:22.417193 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.417158    5820 scope.go:122] "RemoveContainer" containerID="694e1f97c38f20709977d2ba4b28ecb91eee74347f4cc0cbf2c77f8261c5cb07"
Oct 30 19:03:22.417655 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:03:22.417626    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"694e1f97c38f20709977d2ba4b28ecb91eee74347f4cc0cbf2c77f8261c5cb07\": container with ID starting with 694e1f97c38f20709977d2ba4b28ecb91eee74347f4cc0cbf2c77f8261c5cb07 not found: ID does not exist" containerID="694e1f97c38f20709977d2ba4b28ecb91eee74347f4cc0cbf2c77f8261c5cb07"
Oct 30 19:03:22.417764 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.417664    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"694e1f97c38f20709977d2ba4b28ecb91eee74347f4cc0cbf2c77f8261c5cb07"} err="failed to get container status \"694e1f97c38f20709977d2ba4b28ecb91eee74347f4cc0cbf2c77f8261c5cb07\": rpc error: code = NotFound desc = could not find container \"694e1f97c38f20709977d2ba4b28ecb91eee74347f4cc0cbf2c77f8261c5cb07\": container with ID starting with 694e1f97c38f20709977d2ba4b28ecb91eee74347f4cc0cbf2c77f8261c5cb07 not found: ID does not exist"
Oct 30 19:03:22.455412 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.455386    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn"
Oct 30 19:03:22.522971 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.517807    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/efd8ffd1-02a7-4096-8206-de01acd83987-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efd8ffd1-02a7-4096-8206-de01acd83987-plugins-config-volume\") pod \"efd8ffd1-02a7-4096-8206-de01acd83987\" (UID: \"efd8ffd1-02a7-4096-8206-de01acd83987\") "
Oct 30 19:03:22.522971 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.517889    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/efd8ffd1-02a7-4096-8206-de01acd83987-kube-api-access-9qd68\" (UniqueName: \"kubernetes.io/projected/efd8ffd1-02a7-4096-8206-de01acd83987-kube-api-access-9qd68\") pod \"efd8ffd1-02a7-4096-8206-de01acd83987\" (UID: \"efd8ffd1-02a7-4096-8206-de01acd83987\") "
Oct 30 19:03:22.537679 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.537083    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efd8ffd1-02a7-4096-8206-de01acd83987-kube-api-access-9qd68" pod "efd8ffd1-02a7-4096-8206-de01acd83987" (UID: "efd8ffd1-02a7-4096-8206-de01acd83987"). InnerVolumeSpecName "kube-api-access-9qd68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:03:22.538814 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.538641    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qd68\" (UniqueName: \"kubernetes.io/projected/efd8ffd1-02a7-4096-8206-de01acd83987-kube-api-access-9qd68\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.543967 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.540148    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efd8ffd1-02a7-4096-8206-de01acd83987-plugins-config-volume" pod "efd8ffd1-02a7-4096-8206-de01acd83987" (UID: "efd8ffd1-02a7-4096-8206-de01acd83987"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:03:22.547130 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.547104    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:22.564024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.563025    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"]
Oct 30 19:03:22.606189 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.606128    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9vqtg"]
Oct 30 19:03:22.641050 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.641019    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efd8ffd1-02a7-4096-8206-de01acd83987-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.666562 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.666528    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"]
Oct 30 19:03:22.673902 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.673870    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-zfsc6"]
Oct 30 19:03:22.719925 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.719903    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr"
Oct 30 19:03:22.752473 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.752441    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a9f6116-2dd0-4f64-b212-0acf561d5307-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a9f6116-2dd0-4f64-b212-0acf561d5307-plugins-config-volume\") pod \"6a9f6116-2dd0-4f64-b212-0acf561d5307\" (UID: \"6a9f6116-2dd0-4f64-b212-0acf561d5307\") "
Oct 30 19:03:22.792396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.792349    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a9f6116-2dd0-4f64-b212-0acf561d5307-plugins-config-volume" pod "6a9f6116-2dd0-4f64-b212-0acf561d5307" (UID: "6a9f6116-2dd0-4f64-b212-0acf561d5307"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:03:22.792628 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.792402    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a9f6116-2dd0-4f64-b212-0acf561d5307-kube-api-access-4hfrw\" (UniqueName: \"kubernetes.io/projected/6a9f6116-2dd0-4f64-b212-0acf561d5307-kube-api-access-4hfrw\") pod \"6a9f6116-2dd0-4f64-b212-0acf561d5307\" (UID: \"6a9f6116-2dd0-4f64-b212-0acf561d5307\") "
Oct 30 19:03:22.793618 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.793589    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a9f6116-2dd0-4f64-b212-0acf561d5307-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.798413 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.798081    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a9f6116-2dd0-4f64-b212-0acf561d5307-kube-api-access-4hfrw" pod "6a9f6116-2dd0-4f64-b212-0acf561d5307" (UID: "6a9f6116-2dd0-4f64-b212-0acf561d5307"). InnerVolumeSpecName "kube-api-access-4hfrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:03:22.798413 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.798366    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"]
Oct 30 19:03:22.802074 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.801171    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-xtl4b"]
Oct 30 19:03:22.895331 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.895242    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"]
Oct 30 19:03:22.895851 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.895835    5820 cpu_manager.go:414] "containerMap: removing container" podUID="e4ccd454-b219-4d0c-af05-289d90cfe455" containerName="istio-proxy"
Oct 30 19:03:22.895851 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.895850    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e4ccd454-b219-4d0c-af05-289d90cfe455" containerName="istio-proxy"
Oct 30 19:03:22.896025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.895864    5820 cpu_manager.go:414] "containerMap: removing container" podUID="4be0753d-602a-4f4a-a02d-5d6baf686df2" containerName="istio-proxy"
Oct 30 19:03:22.896025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.895871    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4be0753d-602a-4f4a-a02d-5d6baf686df2" containerName="istio-proxy"
Oct 30 19:03:22.896025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.895884    5820 cpu_manager.go:414] "containerMap: removing container" podUID="d2844bd5-87c5-4f3c-86d4-d4041c484355" containerName="istio-proxy"
Oct 30 19:03:22.896025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.895889    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d2844bd5-87c5-4f3c-86d4-d4041c484355" containerName="istio-proxy"
Oct 30 19:03:22.896025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.895901    5820 cpu_manager.go:414] "containerMap: removing container" podUID="370e902d-498f-454d-bacc-1edaf55bcd76" containerName="istio-proxy"
Oct 30 19:03:22.896025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.895905    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="370e902d-498f-454d-bacc-1edaf55bcd76" containerName="istio-proxy"
Oct 30 19:03:22.896025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.895938    5820 cpu_manager.go:414] "containerMap: removing container" podUID="efd8ffd1-02a7-4096-8206-de01acd83987" containerName="payload-processing"
Oct 30 19:03:22.896025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.895943    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="efd8ffd1-02a7-4096-8206-de01acd83987" containerName="payload-processing"
Oct 30 19:03:22.896025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.895972    5820 cpu_manager.go:414] "containerMap: removing container" podUID="6a9f6116-2dd0-4f64-b212-0acf561d5307" containerName="payload-pre-processing"
Oct 30 19:03:22.896025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.895980    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a9f6116-2dd0-4f64-b212-0acf561d5307" containerName="payload-pre-processing"
Oct 30 19:03:22.896025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.895998    5820 cpu_manager.go:414] "containerMap: removing container" podUID="1aac21cb-584a-42ad-9dc6-99aada4cdba1" containerName="istio-proxy"
Oct 30 19:03:22.896025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.896003    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1aac21cb-584a-42ad-9dc6-99aada4cdba1" containerName="istio-proxy"
Oct 30 19:03:22.896418 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.896071    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hfrw\" (UniqueName: \"kubernetes.io/projected/6a9f6116-2dd0-4f64-b212-0acf561d5307-kube-api-access-4hfrw\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.896418 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.896095    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="efd8ffd1-02a7-4096-8206-de01acd83987" containerName="payload-processing"
Oct 30 19:03:22.896418 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.896102    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a9f6116-2dd0-4f64-b212-0acf561d5307" containerName="payload-pre-processing"
Oct 30 19:03:22.896418 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.896112    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="1aac21cb-584a-42ad-9dc6-99aada4cdba1" containerName="istio-proxy"
Oct 30 19:03:22.896418 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.896118    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="d2844bd5-87c5-4f3c-86d4-d4041c484355" containerName="istio-proxy"
Oct 30 19:03:22.896418 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.896124    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="e4ccd454-b219-4d0c-af05-289d90cfe455" containerName="istio-proxy"
Oct 30 19:03:22.896418 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.896133    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="370e902d-498f-454d-bacc-1edaf55bcd76" containerName="istio-proxy"
Oct 30 19:03:22.899511 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.899496    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:22.951705 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.951678    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-a23af5-openshift-default-dockercfg-g5sln\""
Oct 30 19:03:22.963536 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.963506    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"]
Oct 30 19:03:22.997778 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.997742    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-credential-socket\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:22.997975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.997788    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-workload-certs\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:22.997975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.997826    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-workload-socket\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:22.998118 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.998001    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfm8s\" (UniqueName: \"kubernetes.io/projected/8697cc5c-98e3-418b-9d0a-fd6930ce317e-kube-api-access-bfm8s\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:22.998118 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.998074    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-podinfo\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:22.998118 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.998107    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istiod-ca-cert\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:22.998248 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.998146    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-envoy\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:22.998289 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.998269    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-token\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:22.998358 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:22.998340    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-data\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:23.085974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.078553    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:23.163058 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.139821    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-credential-socket\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:23.163058 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.139877    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-workload-certs\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:23.163058 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.139911    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-workload-socket\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:23.163058 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.157443    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-workload-certs\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:23.163058 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.157780    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-credential-socket\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:23.163058 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.158009    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-workload-socket\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:23.177572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.164066    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfm8s\" (UniqueName: \"kubernetes.io/projected/8697cc5c-98e3-418b-9d0a-fd6930ce317e-kube-api-access-bfm8s\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:23.177572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.164167    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-podinfo\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:23.177572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.164201    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istiod-ca-cert\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:23.177572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.164242    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-envoy\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:23.177572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.164307    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-token\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:23.177572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.164382    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-data\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:23.177572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.164891    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-data\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:23.177572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.174006    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-envoy\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:23.177572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.174679    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istiod-ca-cert\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:23.200214 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.200137    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-podinfo\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:23.218022 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.217992    5820 generic.go:356] "Generic (PLEG): container finished" podID="6a9f6116-2dd0-4f64-b212-0acf561d5307" containerID="58c3898193a07c6134acfb024d8f0669218b3c962b3a2a3eba42b86450a31401" exitCode=0
Oct 30 19:03:23.218184 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.218070    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr" event={"ID":"6a9f6116-2dd0-4f64-b212-0acf561d5307","Type":"ContainerDied","Data":"58c3898193a07c6134acfb024d8f0669218b3c962b3a2a3eba42b86450a31401"}
Oct 30 19:03:23.218184 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.218089    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr"
Oct 30 19:03:23.218184 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.218111    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr" event={"ID":"6a9f6116-2dd0-4f64-b212-0acf561d5307","Type":"ContainerDied","Data":"d88eccf037741dd422ad40d1268877481865b7d1707015fe751834781fd4089a"}
Oct 30 19:03:23.218184 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.218139    5820 scope.go:122] "RemoveContainer" containerID="58c3898193a07c6134acfb024d8f0669218b3c962b3a2a3eba42b86450a31401"
Oct 30 19:03:23.221236 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.221211    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn"
Oct 30 19:03:23.221628 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.221603    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn" event={"ID":"efd8ffd1-02a7-4096-8206-de01acd83987","Type":"ContainerDied","Data":"ff05bf57900c1530ac2401e9a92088d51e690bcda11ec35f6006af60d95cc5ce"}
Oct 30 19:03:23.225028 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.223807    5820 generic.go:356] "Generic (PLEG): container finished" podID="efd8ffd1-02a7-4096-8206-de01acd83987" containerID="ff05bf57900c1530ac2401e9a92088d51e690bcda11ec35f6006af60d95cc5ce" exitCode=0
Oct 30 19:03:23.225028 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.223921    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn" event={"ID":"efd8ffd1-02a7-4096-8206-de01acd83987","Type":"ContainerDied","Data":"ebee7b165d8a34338e886088328b0f212d9035c9f91a331fa3d288eb10b652e9"}
Oct 30 19:03:23.235225 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.234044    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfm8s\" (UniqueName: \"kubernetes.io/projected/8697cc5c-98e3-418b-9d0a-fd6930ce317e-kube-api-access-bfm8s\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:23.240822 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.240793    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-token\") pod \"e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") " pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:23.246303 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.245880    5820 scope.go:122] "RemoveContainer" containerID="58c3898193a07c6134acfb024d8f0669218b3c962b3a2a3eba42b86450a31401"
Oct 30 19:03:23.247782 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:03:23.247758    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58c3898193a07c6134acfb024d8f0669218b3c962b3a2a3eba42b86450a31401\": container with ID starting with 58c3898193a07c6134acfb024d8f0669218b3c962b3a2a3eba42b86450a31401 not found: ID does not exist" containerID="58c3898193a07c6134acfb024d8f0669218b3c962b3a2a3eba42b86450a31401"
Oct 30 19:03:23.247909 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.247787    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58c3898193a07c6134acfb024d8f0669218b3c962b3a2a3eba42b86450a31401"} err="failed to get container status \"58c3898193a07c6134acfb024d8f0669218b3c962b3a2a3eba42b86450a31401\": rpc error: code = NotFound desc = could not find container \"58c3898193a07c6134acfb024d8f0669218b3c962b3a2a3eba42b86450a31401\": container with ID starting with 58c3898193a07c6134acfb024d8f0669218b3c962b3a2a3eba42b86450a31401 not found: ID does not exist"
Oct 30 19:03:23.247909 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.247803    5820 scope.go:122] "RemoveContainer" containerID="ff05bf57900c1530ac2401e9a92088d51e690bcda11ec35f6006af60d95cc5ce"
Oct 30 19:03:23.259472 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.259451    5820 scope.go:122] "RemoveContainer" containerID="ff05bf57900c1530ac2401e9a92088d51e690bcda11ec35f6006af60d95cc5ce"
Oct 30 19:03:23.259757 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:03:23.259738    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff05bf57900c1530ac2401e9a92088d51e690bcda11ec35f6006af60d95cc5ce\": container with ID starting with ff05bf57900c1530ac2401e9a92088d51e690bcda11ec35f6006af60d95cc5ce not found: ID does not exist" containerID="ff05bf57900c1530ac2401e9a92088d51e690bcda11ec35f6006af60d95cc5ce"
Oct 30 19:03:23.259806 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.259764    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff05bf57900c1530ac2401e9a92088d51e690bcda11ec35f6006af60d95cc5ce"} err="failed to get container status \"ff05bf57900c1530ac2401e9a92088d51e690bcda11ec35f6006af60d95cc5ce\": rpc error: code = NotFound desc = could not find container \"ff05bf57900c1530ac2401e9a92088d51e690bcda11ec35f6006af60d95cc5ce\": container with ID starting with ff05bf57900c1530ac2401e9a92088d51e690bcda11ec35f6006af60d95cc5ce not found: ID does not exist"
Oct 30 19:03:23.296511 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.296477    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:23.437492 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.437404    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1aac21cb-584a-42ad-9dc6-99aada4cdba1" path="/var/lib/kubelet/pods/1aac21cb-584a-42ad-9dc6-99aada4cdba1/volumes"
Oct 30 19:03:23.438040 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.438018    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="370e902d-498f-454d-bacc-1edaf55bcd76" path="/var/lib/kubelet/pods/370e902d-498f-454d-bacc-1edaf55bcd76/volumes"
Oct 30 19:03:23.438530 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.438511    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e4ccd454-b219-4d0c-af05-289d90cfe455" path="/var/lib/kubelet/pods/e4ccd454-b219-4d0c-af05-289d90cfe455/volumes"
Oct 30 19:03:23.514349 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.514322    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:23.574907 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.574875    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr"]
Oct 30 19:03:23.578750 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.578721    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-77549bd679-2h9cr"]
Oct 30 19:03:23.729785 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.729702    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn"]
Oct 30 19:03:23.732756 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.732730    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-82305a-77bdcb88c8-p74vn"]
Oct 30 19:03:23.789129 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.789095    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:23.949016 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.948991    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"]
Oct 30 19:03:23.950729 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:03:23.950695    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8697cc5c_98e3_418b_9d0a_fd6930ce317e.slice/crio-78509beda30a55bbe454291d335485295c77be0713d98f8125d407a667859b75 WatchSource:0}: Error finding container 78509beda30a55bbe454291d335485295c77be0713d98f8125d407a667859b75: Status 404 returned error can't find the container with id 78509beda30a55bbe454291d335485295c77be0713d98f8125d407a667859b75
Oct 30 19:03:23.953380 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.953346    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:03:23.953490 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.953419    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:03:23.953490 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:23.953463    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:03:24.245465 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:24.245359    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n" event={"ID":"8697cc5c-98e3-418b-9d0a-fd6930ce317e","Type":"ContainerStarted","Data":"e612c5d221069d4238d52d60e6d194e6fa8a4d138763330d9041265d56756db2"}
Oct 30 19:03:24.245935 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:24.245470    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n" event={"ID":"8697cc5c-98e3-418b-9d0a-fd6930ce317e","Type":"ContainerStarted","Data":"78509beda30a55bbe454291d335485295c77be0713d98f8125d407a667859b75"}
Oct 30 19:03:24.369822 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:24.369746    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n" podStartSLOduration=2.369728644 podStartE2EDuration="2.369728644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:03:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:03:24.367802813 +0000 UTC m=+1725.079551511" watchObservedRunningTime="2026-10-30 19:03:24.369728644 +0000 UTC m=+1725.081477343"
Oct 30 19:03:24.516169 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:24.516038    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:25.197987 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:25.196991    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:25.332044 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:25.332014    5820 patch_prober.go:29] interesting pod/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:03:25.332431 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:25.332085    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn" podUID="a98db9fc-3545-4eca-a2b5-964583229a01" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:03:25.437609 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:25.437579    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a9f6116-2dd0-4f64-b212-0acf561d5307" path="/var/lib/kubelet/pods/6a9f6116-2dd0-4f64-b212-0acf561d5307/volumes"
Oct 30 19:03:25.438035 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:25.438020    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="efd8ffd1-02a7-4096-8206-de01acd83987" path="/var/lib/kubelet/pods/efd8ffd1-02a7-4096-8206-de01acd83987/volumes"
Oct 30 19:03:25.520809 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:25.520746    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:26.261747 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:26.261718    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:26.262716 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:26.262692    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:03:26.431764 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:26.431725    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:26.823286 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:26.823253    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:26.984974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:26.982195    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586"]
Oct 30 19:03:26.996977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:26.996196    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586"
Oct 30 19:03:27.024538 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.024505    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk"]
Oct 30 19:03:27.068692 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.062225    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk"
Oct 30 19:03:27.068692 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.063197    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-842bbd\""
Oct 30 19:03:27.068692 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.063538    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pggjw\" (UniqueName: \"kubernetes.io/projected/177456c2-56a4-4d9b-839e-56287b82db19-kube-api-access-pggjw\") pod \"payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586\" (UID: \"177456c2-56a4-4d9b-839e-56287b82db19\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586"
Oct 30 19:03:27.068692 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.063592    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/177456c2-56a4-4d9b-839e-56287b82db19-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586\" (UID: \"177456c2-56a4-4d9b-839e-56287b82db19\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586"
Oct 30 19:03:27.068692 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.063920    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-842bbd-dockercfg-dbp64\""
Oct 30 19:03:27.068692 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.067125    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586"]
Oct 30 19:03:27.068692 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.067189    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk"]
Oct 30 19:03:27.164566 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.164536    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pggjw\" (UniqueName: \"kubernetes.io/projected/177456c2-56a4-4d9b-839e-56287b82db19-kube-api-access-pggjw\") pod \"payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586\" (UID: \"177456c2-56a4-4d9b-839e-56287b82db19\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586"
Oct 30 19:03:27.164724 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.164588    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/177456c2-56a4-4d9b-839e-56287b82db19-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586\" (UID: \"177456c2-56a4-4d9b-839e-56287b82db19\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586"
Oct 30 19:03:27.164724 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.164654    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ec992e5-132c-4d80-9450-8021a3a51e0d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk\" (UID: \"1ec992e5-132c-4d80-9450-8021a3a51e0d\") " pod="openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk"
Oct 30 19:03:27.164724 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.164700    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h94cs\" (UniqueName: \"kubernetes.io/projected/1ec992e5-132c-4d80-9450-8021a3a51e0d-kube-api-access-h94cs\") pod \"payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk\" (UID: \"1ec992e5-132c-4d80-9450-8021a3a51e0d\") " pod="openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk"
Oct 30 19:03:27.165300 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.165279    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/177456c2-56a4-4d9b-839e-56287b82db19-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586\" (UID: \"177456c2-56a4-4d9b-839e-56287b82db19\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586"
Oct 30 19:03:27.225474 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.225445    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pggjw\" (UniqueName: \"kubernetes.io/projected/177456c2-56a4-4d9b-839e-56287b82db19-kube-api-access-pggjw\") pod \"payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586\" (UID: \"177456c2-56a4-4d9b-839e-56287b82db19\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586"
Oct 30 19:03:27.266228 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.266199    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h94cs\" (UniqueName: \"kubernetes.io/projected/1ec992e5-132c-4d80-9450-8021a3a51e0d-kube-api-access-h94cs\") pod \"payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk\" (UID: \"1ec992e5-132c-4d80-9450-8021a3a51e0d\") " pod="openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk"
Oct 30 19:03:27.266435 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.266419    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ec992e5-132c-4d80-9450-8021a3a51e0d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk\" (UID: \"1ec992e5-132c-4d80-9450-8021a3a51e0d\") " pod="openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk"
Oct 30 19:03:27.267182 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.267156    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ec992e5-132c-4d80-9450-8021a3a51e0d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk\" (UID: \"1ec992e5-132c-4d80-9450-8021a3a51e0d\") " pod="openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk"
Oct 30 19:03:27.326130 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.326105    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586"
Oct 30 19:03:27.330348 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.330326    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h94cs\" (UniqueName: \"kubernetes.io/projected/1ec992e5-132c-4d80-9450-8021a3a51e0d-kube-api-access-h94cs\") pod \"payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk\" (UID: \"1ec992e5-132c-4d80-9450-8021a3a51e0d\") " pod="openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk"
Oct 30 19:03:27.395468 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.392271    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk"
Oct 30 19:03:27.662361 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.662328    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 19:03:27.706704 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.706677    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586"]
Oct 30 19:03:27.707542 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:03:27.707520    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod177456c2_56a4_4d9b_839e_56287b82db19.slice/crio-e1484d9193ac7c8ffe1547078c5cf1ae341305768bf8b285d9a106a724a49931 WatchSource:0}: Error finding container e1484d9193ac7c8ffe1547078c5cf1ae341305768bf8b285d9a106a724a49931: Status 404 returned error can't find the container with id e1484d9193ac7c8ffe1547078c5cf1ae341305768bf8b285d9a106a724a49931
Oct 30 19:03:27.773816 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.773785    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a98db9fc-3545-4eca-a2b5-964583229a01-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a98db9fc-3545-4eca-a2b5-964583229a01-istio-podinfo\") pod \"a98db9fc-3545-4eca-a2b5-964583229a01\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") "
Oct 30 19:03:27.773983 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.773855    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a98db9fc-3545-4eca-a2b5-964583229a01-kube-api-access-v6wv4\" (UniqueName: \"kubernetes.io/projected/a98db9fc-3545-4eca-a2b5-964583229a01-kube-api-access-v6wv4\") pod \"a98db9fc-3545-4eca-a2b5-964583229a01\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") "
Oct 30 19:03:27.773983 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.773935    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-workload-certs\") pod \"a98db9fc-3545-4eca-a2b5-964583229a01\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") "
Oct 30 19:03:27.774119 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.774012    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a98db9fc-3545-4eca-a2b5-964583229a01-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a98db9fc-3545-4eca-a2b5-964583229a01-istiod-ca-cert\") pod \"a98db9fc-3545-4eca-a2b5-964583229a01\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") "
Oct 30 19:03:27.774119 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.774079    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-istio-data\") pod \"a98db9fc-3545-4eca-a2b5-964583229a01\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") "
Oct 30 19:03:27.774231 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.774167    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a98db9fc-3545-4eca-a2b5-964583229a01-istio-token\" (UniqueName: \"kubernetes.io/projected/a98db9fc-3545-4eca-a2b5-964583229a01-istio-token\") pod \"a98db9fc-3545-4eca-a2b5-964583229a01\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") "
Oct 30 19:03:27.774287 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.774272    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-credential-socket\") pod \"a98db9fc-3545-4eca-a2b5-964583229a01\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") "
Oct 30 19:03:27.774351 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.774315    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-workload-socket\") pod \"a98db9fc-3545-4eca-a2b5-964583229a01\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") "
Oct 30 19:03:27.774351 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.774345    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-istio-envoy\") pod \"a98db9fc-3545-4eca-a2b5-964583229a01\" (UID: \"a98db9fc-3545-4eca-a2b5-964583229a01\") "
Oct 30 19:03:27.774747 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.774580    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-istio-data" pod "a98db9fc-3545-4eca-a2b5-964583229a01" (UID: "a98db9fc-3545-4eca-a2b5-964583229a01"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:27.774842 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.774776    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-credential-socket" pod "a98db9fc-3545-4eca-a2b5-964583229a01" (UID: "a98db9fc-3545-4eca-a2b5-964583229a01"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:27.775129 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.775103    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:27.775212 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.775134    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:27.775212 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.775163    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a98db9fc-3545-4eca-a2b5-964583229a01-istiod-ca-cert" pod "a98db9fc-3545-4eca-a2b5-964583229a01" (UID: "a98db9fc-3545-4eca-a2b5-964583229a01"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:03:27.775398 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.775237    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-workload-socket" pod "a98db9fc-3545-4eca-a2b5-964583229a01" (UID: "a98db9fc-3545-4eca-a2b5-964583229a01"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:27.775778 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.775754    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-workload-certs" pod "a98db9fc-3545-4eca-a2b5-964583229a01" (UID: "a98db9fc-3545-4eca-a2b5-964583229a01"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:27.776515 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.776490    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a98db9fc-3545-4eca-a2b5-964583229a01-istio-podinfo" pod "a98db9fc-3545-4eca-a2b5-964583229a01" (UID: "a98db9fc-3545-4eca-a2b5-964583229a01"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:03:27.776617 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.776572    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a98db9fc-3545-4eca-a2b5-964583229a01-kube-api-access-v6wv4" pod "a98db9fc-3545-4eca-a2b5-964583229a01" (UID: "a98db9fc-3545-4eca-a2b5-964583229a01"). InnerVolumeSpecName "kube-api-access-v6wv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:03:27.777881 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.777861    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-istio-envoy" pod "a98db9fc-3545-4eca-a2b5-964583229a01" (UID: "a98db9fc-3545-4eca-a2b5-964583229a01"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:27.778019 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.777904    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a98db9fc-3545-4eca-a2b5-964583229a01-istio-token" pod "a98db9fc-3545-4eca-a2b5-964583229a01" (UID: "a98db9fc-3545-4eca-a2b5-964583229a01"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:03:27.805888 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.805865    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk"]
Oct 30 19:03:27.807921 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:03:27.807892    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ec992e5_132c_4d80_9450_8021a3a51e0d.slice/crio-691e6623a3879f88b7adce339ccf59498691a2d762b95ef83f4f8f5f2d48e627 WatchSource:0}: Error finding container 691e6623a3879f88b7adce339ccf59498691a2d762b95ef83f4f8f5f2d48e627: Status 404 returned error can't find the container with id 691e6623a3879f88b7adce339ccf59498691a2d762b95ef83f4f8f5f2d48e627
Oct 30 19:03:27.876453 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.876428    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a98db9fc-3545-4eca-a2b5-964583229a01-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:27.876453 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.876454    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:27.876586 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.876463    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:27.876586 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.876472    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a98db9fc-3545-4eca-a2b5-964583229a01-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:27.876586 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.876481    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6wv4\" (UniqueName: \"kubernetes.io/projected/a98db9fc-3545-4eca-a2b5-964583229a01-kube-api-access-v6wv4\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:27.876586 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.876489    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a98db9fc-3545-4eca-a2b5-964583229a01-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:27.876586 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.876497    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a98db9fc-3545-4eca-a2b5-964583229a01-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:27.901232 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:27.901169    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:28.168495 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:28.168399    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:28.286980 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:28.286919    5820 generic.go:356] "Generic (PLEG): container finished" podID="a98db9fc-3545-4eca-a2b5-964583229a01" containerID="d42168379ca9c4657fbfecb07c367460e0ccc4dc1ffcc083a40c0bc38cd86ad2" exitCode=0
Oct 30 19:03:28.287155 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:28.287001    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"
Oct 30 19:03:28.287155 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:28.286998    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn" event={"ID":"a98db9fc-3545-4eca-a2b5-964583229a01","Type":"ContainerDied","Data":"d42168379ca9c4657fbfecb07c367460e0ccc4dc1ffcc083a40c0bc38cd86ad2"}
Oct 30 19:03:28.287155 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:28.287073    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn" event={"ID":"a98db9fc-3545-4eca-a2b5-964583229a01","Type":"ContainerDied","Data":"4a3495fcb90240952601482bb0d34fc801dd348c6faf85c66e3e99c0f4061e71"}
Oct 30 19:03:28.287155 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:28.287098    5820 scope.go:122] "RemoveContainer" containerID="d42168379ca9c4657fbfecb07c367460e0ccc4dc1ffcc083a40c0bc38cd86ad2"
Oct 30 19:03:28.289026 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:28.288997    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk" event={"ID":"1ec992e5-132c-4d80-9450-8021a3a51e0d","Type":"ContainerStarted","Data":"bf16c4ee371e3f724e847ed9b97b21df67429b77af59c4be48b6215c194dbe66"}
Oct 30 19:03:28.289137 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:28.289048    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk" event={"ID":"1ec992e5-132c-4d80-9450-8021a3a51e0d","Type":"ContainerStarted","Data":"691e6623a3879f88b7adce339ccf59498691a2d762b95ef83f4f8f5f2d48e627"}
Oct 30 19:03:28.289194 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:28.289152    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk"
Oct 30 19:03:28.291492 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:28.291471    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586" event={"ID":"177456c2-56a4-4d9b-839e-56287b82db19","Type":"ContainerStarted","Data":"7b700f0ee723b402386e9889a0abdf5b4e24ed4304b13148291d100956e07dbd"}
Oct 30 19:03:28.291603 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:28.291512    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586" event={"ID":"177456c2-56a4-4d9b-839e-56287b82db19","Type":"ContainerStarted","Data":"e1484d9193ac7c8ffe1547078c5cf1ae341305768bf8b285d9a106a724a49931"}
Oct 30 19:03:28.291603 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:28.291565    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586"
Oct 30 19:03:28.298239 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:28.298225    5820 scope.go:122] "RemoveContainer" containerID="b4c5572aa1125e96062decafb7d2efc601feb6bcce7e1b92f90675408f277fbd"
Oct 30 19:03:28.307127 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:28.307108    5820 scope.go:122] "RemoveContainer" containerID="d42168379ca9c4657fbfecb07c367460e0ccc4dc1ffcc083a40c0bc38cd86ad2"
Oct 30 19:03:28.307380 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:03:28.307362    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d42168379ca9c4657fbfecb07c367460e0ccc4dc1ffcc083a40c0bc38cd86ad2\": container with ID starting with d42168379ca9c4657fbfecb07c367460e0ccc4dc1ffcc083a40c0bc38cd86ad2 not found: ID does not exist" containerID="d42168379ca9c4657fbfecb07c367460e0ccc4dc1ffcc083a40c0bc38cd86ad2"
Oct 30 19:03:28.307456 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:28.307394    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d42168379ca9c4657fbfecb07c367460e0ccc4dc1ffcc083a40c0bc38cd86ad2"} err="failed to get container status \"d42168379ca9c4657fbfecb07c367460e0ccc4dc1ffcc083a40c0bc38cd86ad2\": rpc error: code = NotFound desc = could not find container \"d42168379ca9c4657fbfecb07c367460e0ccc4dc1ffcc083a40c0bc38cd86ad2\": container with ID starting with d42168379ca9c4657fbfecb07c367460e0ccc4dc1ffcc083a40c0bc38cd86ad2 not found: ID does not exist"
Oct 30 19:03:28.307456 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:28.307418    5820 scope.go:122] "RemoveContainer" containerID="b4c5572aa1125e96062decafb7d2efc601feb6bcce7e1b92f90675408f277fbd"
Oct 30 19:03:28.307655 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:03:28.307636    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4c5572aa1125e96062decafb7d2efc601feb6bcce7e1b92f90675408f277fbd\": container with ID starting with b4c5572aa1125e96062decafb7d2efc601feb6bcce7e1b92f90675408f277fbd not found: ID does not exist" containerID="b4c5572aa1125e96062decafb7d2efc601feb6bcce7e1b92f90675408f277fbd"
Oct 30 19:03:28.307700 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:28.307661    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4c5572aa1125e96062decafb7d2efc601feb6bcce7e1b92f90675408f277fbd"} err="failed to get container status \"b4c5572aa1125e96062decafb7d2efc601feb6bcce7e1b92f90675408f277fbd\": rpc error: code = NotFound desc = could not find container \"b4c5572aa1125e96062decafb7d2efc601feb6bcce7e1b92f90675408f277fbd\": container with ID starting with b4c5572aa1125e96062decafb7d2efc601feb6bcce7e1b92f90675408f277fbd not found: ID does not exist"
Oct 30 19:03:28.482664 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:28.482564    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk" podStartSLOduration=2.4825507 podStartE2EDuration="2.4825507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:03:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:03:28.481885314 +0000 UTC m=+1729.193634011" watchObservedRunningTime="2026-10-30 19:03:28.4825507 +0000 UTC m=+1729.194299398"
Oct 30 19:03:28.619809 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:28.607025    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"]
Oct 30 19:03:28.657300 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:28.657235    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-83d5bd-openshift-default-788d5c45b9-f97jn"]
Oct 30 19:03:28.747712 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:28.747593    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586" podStartSLOduration=2.747578748 podStartE2EDuration="2.747578748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:03:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:03:28.745420646 +0000 UTC m=+1729.457169343" watchObservedRunningTime="2026-10-30 19:03:28.747578748 +0000 UTC m=+1729.459327494"
Oct 30 19:03:29.437293 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:29.437260    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a98db9fc-3545-4eca-a2b5-964583229a01" path="/var/lib/kubelet/pods/a98db9fc-3545-4eca-a2b5-964583229a01/volumes"
Oct 30 19:03:30.639224 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:30.639183    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:31.454060 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:31.454022    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:31.687328 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:31.687288    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:31.992812 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:31.992774    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx"]
Oct 30 19:03:31.994035 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:31.994006    5820 cpu_manager.go:414] "containerMap: removing container" podUID="a98db9fc-3545-4eca-a2b5-964583229a01" containerName="istio-proxy"
Oct 30 19:03:31.994179 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:31.994166    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a98db9fc-3545-4eca-a2b5-964583229a01" containerName="istio-proxy"
Oct 30 19:03:31.994496 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:31.994478    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="a98db9fc-3545-4eca-a2b5-964583229a01" containerName="istio-proxy"
Oct 30 19:03:31.994639 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:31.994625    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="a98db9fc-3545-4eca-a2b5-964583229a01" containerName="istio-proxy"
Oct 30 19:03:32.041983 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.028330    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx"
Oct 30 19:03:32.056325 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.056294    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx"]
Oct 30 19:03:32.065281 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.065240    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx\" (UID: \"408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx"
Oct 30 19:03:32.065478 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.065456    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gpbz\" (UniqueName: \"kubernetes.io/projected/408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe-kube-api-access-5gpbz\") pod \"payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx\" (UID: \"408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx"
Oct 30 19:03:32.094799 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.094768    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-a23af5\""
Oct 30 19:03:32.095104 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.095081    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-a23af5-dockercfg-jtgwc\""
Oct 30 19:03:32.134041 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.134008    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k"]
Oct 30 19:03:32.134695 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.134677    5820 cpu_manager.go:414] "containerMap: removing container" podUID="a98db9fc-3545-4eca-a2b5-964583229a01" containerName="istio-proxy"
Oct 30 19:03:32.134805 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.134698    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a98db9fc-3545-4eca-a2b5-964583229a01" containerName="istio-proxy"
Oct 30 19:03:32.138656 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.138631    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k"
Oct 30 19:03:32.166328 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.166305    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gpbz\" (UniqueName: \"kubernetes.io/projected/408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe-kube-api-access-5gpbz\") pod \"payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx\" (UID: \"408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx"
Oct 30 19:03:32.166484 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.166352    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbwcn\" (UniqueName: \"kubernetes.io/projected/e4b29a58-09b8-4ef8-be55-49dcac8dc6a1-kube-api-access-jbwcn\") pod \"payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k\" (UID: \"e4b29a58-09b8-4ef8-be55-49dcac8dc6a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k"
Oct 30 19:03:32.166484 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.166397    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4b29a58-09b8-4ef8-be55-49dcac8dc6a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k\" (UID: \"e4b29a58-09b8-4ef8-be55-49dcac8dc6a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k"
Oct 30 19:03:32.166484 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.166462    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx\" (UID: \"408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx"
Oct 30 19:03:32.167141 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.167124    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx\" (UID: \"408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx"
Oct 30 19:03:32.185497 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.185468    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k"]
Oct 30 19:03:32.229847 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.229816    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gpbz\" (UniqueName: \"kubernetes.io/projected/408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe-kube-api-access-5gpbz\") pod \"payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx\" (UID: \"408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx"
Oct 30 19:03:32.267606 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.267542    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbwcn\" (UniqueName: \"kubernetes.io/projected/e4b29a58-09b8-4ef8-be55-49dcac8dc6a1-kube-api-access-jbwcn\") pod \"payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k\" (UID: \"e4b29a58-09b8-4ef8-be55-49dcac8dc6a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k"
Oct 30 19:03:32.267606 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.267602    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4b29a58-09b8-4ef8-be55-49dcac8dc6a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k\" (UID: \"e4b29a58-09b8-4ef8-be55-49dcac8dc6a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k"
Oct 30 19:03:32.268302 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.268279    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4b29a58-09b8-4ef8-be55-49dcac8dc6a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k\" (UID: \"e4b29a58-09b8-4ef8-be55-49dcac8dc6a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k"
Oct 30 19:03:32.328434 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.328406    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbwcn\" (UniqueName: \"kubernetes.io/projected/e4b29a58-09b8-4ef8-be55-49dcac8dc6a1-kube-api-access-jbwcn\") pod \"payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k\" (UID: \"e4b29a58-09b8-4ef8-be55-49dcac8dc6a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k"
Oct 30 19:03:32.393082 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.393050    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx"
Oct 30 19:03:32.478978 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.472663    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k"
Oct 30 19:03:32.821737 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.821708    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx"]
Oct 30 19:03:32.822863 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:03:32.822834    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod408a2e0d_e13c_4f8a_8f3b_9d16db19d9fe.slice/crio-0eadef1f993d1e496d14949a0c239ea2654c942630d3cb47dea6fd66bd0e74dc WatchSource:0}: Error finding container 0eadef1f993d1e496d14949a0c239ea2654c942630d3cb47dea6fd66bd0e74dc: Status 404 returned error can't find the container with id 0eadef1f993d1e496d14949a0c239ea2654c942630d3cb47dea6fd66bd0e74dc
Oct 30 19:03:32.885717 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:03:32.885684    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4b29a58_09b8_4ef8_be55_49dcac8dc6a1.slice/crio-25092f91ec56db4ecf797783659340c34638d16727805bb2f67342e86162e84e WatchSource:0}: Error finding container 25092f91ec56db4ecf797783659340c34638d16727805bb2f67342e86162e84e: Status 404 returned error can't find the container with id 25092f91ec56db4ecf797783659340c34638d16727805bb2f67342e86162e84e
Oct 30 19:03:32.885985 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:32.885927    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k"]
Oct 30 19:03:33.329991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:33.329938    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k" event={"ID":"e4b29a58-09b8-4ef8-be55-49dcac8dc6a1","Type":"ContainerStarted","Data":"975214c158c7ffb3869f47fce67a66eb7807c4be139cf9eef1c9f22d5c1638da"}
Oct 30 19:03:33.330225 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:33.330015    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k" event={"ID":"e4b29a58-09b8-4ef8-be55-49dcac8dc6a1","Type":"ContainerStarted","Data":"25092f91ec56db4ecf797783659340c34638d16727805bb2f67342e86162e84e"}
Oct 30 19:03:33.330225 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:33.330044    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k"
Oct 30 19:03:33.331593 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:33.331571    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx" event={"ID":"408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe","Type":"ContainerStarted","Data":"49abf79390b38852c33b2d6ebcee2377deb53f80bd940cd8bfa82ae1714b44e5"}
Oct 30 19:03:33.331682 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:33.331609    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx" event={"ID":"408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe","Type":"ContainerStarted","Data":"0eadef1f993d1e496d14949a0c239ea2654c942630d3cb47dea6fd66bd0e74dc"}
Oct 30 19:03:33.331682 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:33.331640    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx"
Oct 30 19:03:33.447623 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:33.447454    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k" podStartSLOduration=1.4474373329999999 podStartE2EDuration="1.447437333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:03:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:03:33.447017308 +0000 UTC m=+1734.158766006" watchObservedRunningTime="2026-10-30 19:03:33.447437333 +0000 UTC m=+1734.159186031"
Oct 30 19:03:35.519974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:35.516750    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:35.771778 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:35.771691    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:36.065693 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:36.065657    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:38.204106 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:03:38.204070    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:03:39.302321 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:39.302293    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk"
Oct 30 19:03:39.302703 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:39.302358    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586"
Oct 30 19:03:39.422408 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:39.422353    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx" podStartSLOduration=8.422341011 podStartE2EDuration="8.422341011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:03:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:03:33.594262048 +0000 UTC m=+1734.306010747" watchObservedRunningTime="2026-10-30 19:03:39.422341011 +0000 UTC m=+1740.134089708"
Oct 30 19:03:40.805067 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:40.804973    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:44.341996 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:44.341941    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx"
Oct 30 19:03:44.342597 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:44.342052    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k"
Oct 30 19:03:45.591434 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:45.591403    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:50.615861 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:50.615824    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:54.143085 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:54.143046    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:54.651732 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:54.651692    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:55.714112 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:55.714079    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x"]
Oct 30 19:03:55.719841 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:55.719822    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x"
Oct 30 19:03:55.730484 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:55.730461    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x\" (UID: \"fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x"
Oct 30 19:03:55.730598 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:55.730582    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gllg\" (UniqueName: \"kubernetes.io/projected/fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d-kube-api-access-5gllg\") pod \"payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x\" (UID: \"fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x"
Oct 30 19:03:55.777150 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:55.777119    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x"]
Oct 30 19:03:55.822999 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:55.822971    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-82305a\""
Oct 30 19:03:55.823175 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:55.823149    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-82305a-dockercfg-pbss9\""
Oct 30 19:03:55.832326 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:55.832299    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x\" (UID: \"fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x"
Oct 30 19:03:55.832441 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:55.832395    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gllg\" (UniqueName: \"kubernetes.io/projected/fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d-kube-api-access-5gllg\") pod \"payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x\" (UID: \"fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x"
Oct 30 19:03:55.833020 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:55.832999    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x\" (UID: \"fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x"
Oct 30 19:03:55.893273 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:55.893248    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gllg\" (UniqueName: \"kubernetes.io/projected/fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d-kube-api-access-5gllg\") pod \"payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x\" (UID: \"fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x"
Oct 30 19:03:56.033635 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:56.033564    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x"
Oct 30 19:03:56.262314 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:56.262277    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:56.409712 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:56.409682    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x"]
Oct 30 19:03:56.411424 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:03:56.411400    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe43b403_5fe6_4f1b_a2fc_9bfde561ca7d.slice/crio-e66c4e1be7025daab4e104cb1bc28a1b438671cc8b9062b4334627129f1df01a WatchSource:0}: Error finding container e66c4e1be7025daab4e104cb1bc28a1b438671cc8b9062b4334627129f1df01a: Status 404 returned error can't find the container with id e66c4e1be7025daab4e104cb1bc28a1b438671cc8b9062b4334627129f1df01a
Oct 30 19:03:56.454270 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:56.454239    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:56.477332 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:56.477300    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x" event={"ID":"fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d","Type":"ContainerStarted","Data":"e66c4e1be7025daab4e104cb1bc28a1b438671cc8b9062b4334627129f1df01a"}
Oct 30 19:03:56.491132 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:56.491111    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg"]
Oct 30 19:03:56.496561 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:56.496538    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg"
Oct 30 19:03:56.539386 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:56.539363    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db22814d-f854-4959-8f90-3c0e667ab59b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg\" (UID: \"db22814d-f854-4959-8f90-3c0e667ab59b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg"
Oct 30 19:03:56.539484 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:56.539396    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdbdl\" (UniqueName: \"kubernetes.io/projected/db22814d-f854-4959-8f90-3c0e667ab59b-kube-api-access-kdbdl\") pod \"payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg\" (UID: \"db22814d-f854-4959-8f90-3c0e667ab59b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg"
Oct 30 19:03:56.553515 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:56.553490    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg"]
Oct 30 19:03:56.640494 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:56.640463    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db22814d-f854-4959-8f90-3c0e667ab59b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg\" (UID: \"db22814d-f854-4959-8f90-3c0e667ab59b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg"
Oct 30 19:03:56.640642 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:56.640501    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdbdl\" (UniqueName: \"kubernetes.io/projected/db22814d-f854-4959-8f90-3c0e667ab59b-kube-api-access-kdbdl\") pod \"payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg\" (UID: \"db22814d-f854-4959-8f90-3c0e667ab59b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg"
Oct 30 19:03:56.641167 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:56.641148    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db22814d-f854-4959-8f90-3c0e667ab59b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg\" (UID: \"db22814d-f854-4959-8f90-3c0e667ab59b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg"
Oct 30 19:03:56.701683 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:56.701628    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdbdl\" (UniqueName: \"kubernetes.io/projected/db22814d-f854-4959-8f90-3c0e667ab59b-kube-api-access-kdbdl\") pod \"payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg\" (UID: \"db22814d-f854-4959-8f90-3c0e667ab59b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg"
Oct 30 19:03:56.811286 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:56.811248    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg"
Oct 30 19:03:57.190295 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:57.190267    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg"]
Oct 30 19:03:57.191693 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:03:57.191662    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb22814d_f854_4959_8f90_3c0e667ab59b.slice/crio-75e0c4a34eb37d8ce88b8eb2d174667a00960f5c274c4502098211918cf83250 WatchSource:0}: Error finding container 75e0c4a34eb37d8ce88b8eb2d174667a00960f5c274c4502098211918cf83250: Status 404 returned error can't find the container with id 75e0c4a34eb37d8ce88b8eb2d174667a00960f5c274c4502098211918cf83250
Oct 30 19:03:57.422979 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:57.422050    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:57.489130 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:03:57.489042    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg" event={"ID":"db22814d-f854-4959-8f90-3c0e667ab59b","Type":"ContainerStarted","Data":"75e0c4a34eb37d8ce88b8eb2d174667a00960f5c274c4502098211918cf83250"}
Oct 30 19:04:01.519199 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:01.519092    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg" event={"ID":"db22814d-f854-4959-8f90-3c0e667ab59b","Type":"ContainerStarted","Data":"bfadc79342a99bebf8fae041524878f319f1390f3eb591fcf0321dda0c316738"}
Oct 30 19:04:01.519598 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:01.519256    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg"
Oct 30 19:04:01.520776 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:01.520743    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x" event={"ID":"fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d","Type":"ContainerStarted","Data":"65addbcc80b735c26b7609316dff1e5fbd59611012d830b58ae0e5e519e09bce"}
Oct 30 19:04:01.520913 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:01.520806    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x"
Oct 30 19:04:01.674785 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:01.674696    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg" podStartSLOduration=1.651318126 podStartE2EDuration="5.674681218s" totalImagesPullingTime="4.023363092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:03:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:04:01.670285483 +0000 UTC m=+1762.382034204" watchObservedRunningTime="2026-10-30 19:04:01.674681218 +0000 UTC m=+1762.386429916"
Oct 30 19:04:01.802037 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:01.801107    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x" podStartSLOduration=1.985315822 podStartE2EDuration="6.801085824s" totalImagesPullingTime="4.815770002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:03:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:04:01.793675372 +0000 UTC m=+1762.505424070" watchObservedRunningTime="2026-10-30 19:04:01.801085824 +0000 UTC m=+1762.512834523"
Oct 30 19:04:01.850736 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:01.850703    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"]
Oct 30 19:04:01.855402 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:01.855361    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"
Oct 30 19:04:01.910440 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:01.910241    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"]
Oct 30 19:04:01.924003 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:01.911158    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-842bbd-serving-cert\""
Oct 30 19:04:01.924003 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:01.911982    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-metrics-certs\") pod \"maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg\" (UID: \"727362a7-2b8f-401d-afbd-71a8dcc2fd7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"
Oct 30 19:04:01.924003 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:01.912070    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-openshift-service-ca\") pod \"maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg\" (UID: \"727362a7-2b8f-401d-afbd-71a8dcc2fd7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"
Oct 30 19:04:01.924003 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:01.912129    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-maas-api-tls\") pod \"maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg\" (UID: \"727362a7-2b8f-401d-afbd-71a8dcc2fd7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"
Oct 30 19:04:01.924003 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:01.912325    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2nxq\" (UniqueName: \"kubernetes.io/projected/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-kube-api-access-g2nxq\") pod \"maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg\" (UID: \"727362a7-2b8f-401d-afbd-71a8dcc2fd7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"
Oct 30 19:04:02.023096 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:02.019613    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-maas-api-tls\") pod \"maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg\" (UID: \"727362a7-2b8f-401d-afbd-71a8dcc2fd7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"
Oct 30 19:04:02.023096 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:02.019845    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2nxq\" (UniqueName: \"kubernetes.io/projected/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-kube-api-access-g2nxq\") pod \"maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg\" (UID: \"727362a7-2b8f-401d-afbd-71a8dcc2fd7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"
Oct 30 19:04:02.023096 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:02.019922    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-metrics-certs\") pod \"maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg\" (UID: \"727362a7-2b8f-401d-afbd-71a8dcc2fd7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"
Oct 30 19:04:02.023096 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:02.020007    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-openshift-service-ca\") pod \"maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg\" (UID: \"727362a7-2b8f-401d-afbd-71a8dcc2fd7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"
Oct 30 19:04:02.023096 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:04:02.020285    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-serving-cert: secret "maas-api-e2e-api-a-842bbd-serving-cert" not found
Oct 30 19:04:02.023096 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:04:02.020352    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-maas-api-tls podName:727362a7-2b8f-401d-afbd-71a8dcc2fd7f nodeName:}" failed. No retries permitted until 2026-10-30 19:04:02.520330945 +0000 UTC m=+1763.232079644 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-maas-api-tls") pod "maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg" (UID: "727362a7-2b8f-401d-afbd-71a8dcc2fd7f") : secret "maas-api-e2e-api-a-842bbd-serving-cert" not found
Oct 30 19:04:02.023096 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:02.020820    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-openshift-service-ca\") pod \"maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg\" (UID: \"727362a7-2b8f-401d-afbd-71a8dcc2fd7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"
Oct 30 19:04:02.030221 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:02.024683    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-metrics-certs\") pod \"maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg\" (UID: \"727362a7-2b8f-401d-afbd-71a8dcc2fd7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"
Oct 30 19:04:02.083309 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:02.082849    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2nxq\" (UniqueName: \"kubernetes.io/projected/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-kube-api-access-g2nxq\") pod \"maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg\" (UID: \"727362a7-2b8f-401d-afbd-71a8dcc2fd7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"
Oct 30 19:04:02.145989 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:02.142474    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:02.381148 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:02.380759    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:02.533226 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:02.533192    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-maas-api-tls\") pod \"maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg\" (UID: \"727362a7-2b8f-401d-afbd-71a8dcc2fd7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"
Oct 30 19:04:02.536037 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:02.536005    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-maas-api-tls\") pod \"maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg\" (UID: \"727362a7-2b8f-401d-afbd-71a8dcc2fd7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"
Oct 30 19:04:02.610119 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:02.610085    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:02.770061 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:02.769967    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"
Oct 30 19:04:03.173860 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:03.173822    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"]
Oct 30 19:04:03.175871 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:04:03.175835    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod727362a7_2b8f_401d_afbd_71a8dcc2fd7f.slice/crio-51df928a98832872f18039e91cfef8e217cd728d5411684694b8e38e02a72205 WatchSource:0}: Error finding container 51df928a98832872f18039e91cfef8e217cd728d5411684694b8e38e02a72205: Status 404 returned error can't find the container with id 51df928a98832872f18039e91cfef8e217cd728d5411684694b8e38e02a72205
Oct 30 19:04:03.534673 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:03.534591    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg" event={"ID":"727362a7-2b8f-401d-afbd-71a8dcc2fd7f","Type":"ContainerStarted","Data":"51df928a98832872f18039e91cfef8e217cd728d5411684694b8e38e02a72205"}
Oct 30 19:04:04.543731 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:04.543643    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg" event={"ID":"727362a7-2b8f-401d-afbd-71a8dcc2fd7f","Type":"ContainerStarted","Data":"f7ac85c163ab31b1a5a2771728303aaa3d1ef44caf225ee09e5b187e45f08d27"}
Oct 30 19:04:04.544225 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:04.543742    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"
Oct 30 19:04:04.667346 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:04.667295    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg" podStartSLOduration=2.689171221 podStartE2EDuration="3.66728146s" totalImagesPullingTime="978.110239ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:04:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:04:04.665321745 +0000 UTC m=+1765.377070453" watchObservedRunningTime="2026-10-30 19:04:04.66728146 +0000 UTC m=+1765.379030157"
Oct 30 19:04:06.296934 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:06.296893    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"]
Oct 30 19:04:06.297467 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:06.297168    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282" podUID="1154f650-593c-4ae1-b703-61d4b70fc24b" containerName="maas-api" containerID="cri-o://584c31620e5afab493102ca988c419d9db897e0841b67ddaecaae05c96fe6b3c" gracePeriod=30
Oct 30 19:04:06.430803 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:06.430755    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:06.593492 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:06.593449    5820 generic.go:356] "Generic (PLEG): container finished" podID="1154f650-593c-4ae1-b703-61d4b70fc24b" containerID="584c31620e5afab493102ca988c419d9db897e0841b67ddaecaae05c96fe6b3c" exitCode=0
Oct 30 19:04:06.593672 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:06.593593    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282" event={"ID":"1154f650-593c-4ae1-b703-61d4b70fc24b","Type":"ContainerDied","Data":"584c31620e5afab493102ca988c419d9db897e0841b67ddaecaae05c96fe6b3c"}
Oct 30 19:04:06.636314 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:06.636284    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"
Oct 30 19:04:06.693652 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:06.693615    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1154f650-593c-4ae1-b703-61d4b70fc24b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1154f650-593c-4ae1-b703-61d4b70fc24b-maas-api-tls\") pod \"1154f650-593c-4ae1-b703-61d4b70fc24b\" (UID: \"1154f650-593c-4ae1-b703-61d4b70fc24b\") "
Oct 30 19:04:06.695206 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:06.694006    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1154f650-593c-4ae1-b703-61d4b70fc24b-kube-api-access-qrffh\" (UniqueName: \"kubernetes.io/projected/1154f650-593c-4ae1-b703-61d4b70fc24b-kube-api-access-qrffh\") pod \"1154f650-593c-4ae1-b703-61d4b70fc24b\" (UID: \"1154f650-593c-4ae1-b703-61d4b70fc24b\") "
Oct 30 19:04:06.695634 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:06.695610    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1154f650-593c-4ae1-b703-61d4b70fc24b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1154f650-593c-4ae1-b703-61d4b70fc24b-metrics-certs\") pod \"1154f650-593c-4ae1-b703-61d4b70fc24b\" (UID: \"1154f650-593c-4ae1-b703-61d4b70fc24b\") "
Oct 30 19:04:06.696014 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:06.695872    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1154f650-593c-4ae1-b703-61d4b70fc24b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1154f650-593c-4ae1-b703-61d4b70fc24b-openshift-service-ca\") pod \"1154f650-593c-4ae1-b703-61d4b70fc24b\" (UID: \"1154f650-593c-4ae1-b703-61d4b70fc24b\") "
Oct 30 19:04:06.697864 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:06.697811    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1154f650-593c-4ae1-b703-61d4b70fc24b-openshift-service-ca" pod "1154f650-593c-4ae1-b703-61d4b70fc24b" (UID: "1154f650-593c-4ae1-b703-61d4b70fc24b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:04:06.703609 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:06.703566    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1154f650-593c-4ae1-b703-61d4b70fc24b-kube-api-access-qrffh" pod "1154f650-593c-4ae1-b703-61d4b70fc24b" (UID: "1154f650-593c-4ae1-b703-61d4b70fc24b"). InnerVolumeSpecName "kube-api-access-qrffh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:04:06.704256 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:06.704214    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1154f650-593c-4ae1-b703-61d4b70fc24b-maas-api-tls" pod "1154f650-593c-4ae1-b703-61d4b70fc24b" (UID: "1154f650-593c-4ae1-b703-61d4b70fc24b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:04:06.740729 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:06.740668    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1154f650-593c-4ae1-b703-61d4b70fc24b-metrics-certs" pod "1154f650-593c-4ae1-b703-61d4b70fc24b" (UID: "1154f650-593c-4ae1-b703-61d4b70fc24b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:04:06.798275 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:06.798245    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1154f650-593c-4ae1-b703-61d4b70fc24b-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:04:06.798275 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:06.798273    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1154f650-593c-4ae1-b703-61d4b70fc24b-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:04:06.798275 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:06.798283    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrffh\" (UniqueName: \"kubernetes.io/projected/1154f650-593c-4ae1-b703-61d4b70fc24b-kube-api-access-qrffh\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:04:06.798517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:06.798294    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1154f650-593c-4ae1-b703-61d4b70fc24b-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:04:07.601214 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:07.601184    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"
Oct 30 19:04:07.601214 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:07.601210    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282" event={"ID":"1154f650-593c-4ae1-b703-61d4b70fc24b","Type":"ContainerDied","Data":"686764cb4f19f0bf35b0e617557e0e0b1bf8559e2fc5eac804a5eb3c21b9ff91"}
Oct 30 19:04:07.601696 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:07.601264    5820 scope.go:122] "RemoveContainer" containerID="584c31620e5afab493102ca988c419d9db897e0841b67ddaecaae05c96fe6b3c"
Oct 30 19:04:07.765595 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:07.765519    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"]
Oct 30 19:04:07.815030 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:07.815001    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-132af1-697db78c58-d8282"]
Oct 30 19:04:07.953229 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:07.953193    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:08.203810 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:04:08.203772    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:04:08.244048 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:08.244015    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:08.446929 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:08.446865    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:08.901101 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:08.901045    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:09.437538 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:09.437504    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1154f650-593c-4ae1-b703-61d4b70fc24b" path="/var/lib/kubelet/pods/1154f650-593c-4ae1-b703-61d4b70fc24b/volumes"
Oct 30 19:04:10.174913 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:10.174870    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:10.380721 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:10.380685    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:10.555491 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:10.555468    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"
Oct 30 19:04:12.529574 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:12.529544    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x"
Oct 30 19:04:12.530004 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:12.529615    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg"
Oct 30 19:04:13.230258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.230220    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"]
Oct 30 19:04:13.231009 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.230990    5820 cpu_manager.go:414] "containerMap: removing container" podUID="1154f650-593c-4ae1-b703-61d4b70fc24b" containerName="maas-api"
Oct 30 19:04:13.231065 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.231011    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1154f650-593c-4ae1-b703-61d4b70fc24b" containerName="maas-api"
Oct 30 19:04:13.231139 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.231130    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="1154f650-593c-4ae1-b703-61d4b70fc24b" containerName="maas-api"
Oct 30 19:04:13.236169 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.236149    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"
Oct 30 19:04:13.290296 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.290269    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-a23af5-serving-cert\""
Oct 30 19:04:13.291185 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.291127    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47e07161-1cfe-4075-84fb-52a095c05533-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv\" (UID: \"47e07161-1cfe-4075-84fb-52a095c05533\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"
Oct 30 19:04:13.291319 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.291300    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47e07161-1cfe-4075-84fb-52a095c05533-metrics-certs\") pod \"maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv\" (UID: \"47e07161-1cfe-4075-84fb-52a095c05533\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"
Oct 30 19:04:13.291463 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.291379    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47e07161-1cfe-4075-84fb-52a095c05533-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv\" (UID: \"47e07161-1cfe-4075-84fb-52a095c05533\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"
Oct 30 19:04:13.291568 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.291546    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cgzn\" (UniqueName: \"kubernetes.io/projected/47e07161-1cfe-4075-84fb-52a095c05533-kube-api-access-5cgzn\") pod \"maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv\" (UID: \"47e07161-1cfe-4075-84fb-52a095c05533\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"
Oct 30 19:04:13.295589 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.295565    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"]
Oct 30 19:04:13.392738 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.392703    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47e07161-1cfe-4075-84fb-52a095c05533-metrics-certs\") pod \"maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv\" (UID: \"47e07161-1cfe-4075-84fb-52a095c05533\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"
Oct 30 19:04:13.392920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.392768    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47e07161-1cfe-4075-84fb-52a095c05533-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv\" (UID: \"47e07161-1cfe-4075-84fb-52a095c05533\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"
Oct 30 19:04:13.392920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.392834    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cgzn\" (UniqueName: \"kubernetes.io/projected/47e07161-1cfe-4075-84fb-52a095c05533-kube-api-access-5cgzn\") pod \"maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv\" (UID: \"47e07161-1cfe-4075-84fb-52a095c05533\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"
Oct 30 19:04:13.392920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.392914    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47e07161-1cfe-4075-84fb-52a095c05533-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv\" (UID: \"47e07161-1cfe-4075-84fb-52a095c05533\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"
Oct 30 19:04:13.393113 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:04:13.393045    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-serving-cert: secret "maas-api-e2e-ipp-a-a23af5-serving-cert" not found
Oct 30 19:04:13.393164 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:04:13.393141    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47e07161-1cfe-4075-84fb-52a095c05533-maas-api-tls podName:47e07161-1cfe-4075-84fb-52a095c05533 nodeName:}" failed. No retries permitted until 2026-10-30 19:04:13.893124202 +0000 UTC m=+1774.604872882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/47e07161-1cfe-4075-84fb-52a095c05533-maas-api-tls") pod "maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv" (UID: "47e07161-1cfe-4075-84fb-52a095c05533") : secret "maas-api-e2e-ipp-a-a23af5-serving-cert" not found
Oct 30 19:04:13.393551 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.393532    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47e07161-1cfe-4075-84fb-52a095c05533-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv\" (UID: \"47e07161-1cfe-4075-84fb-52a095c05533\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"
Oct 30 19:04:13.395309 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.395291    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47e07161-1cfe-4075-84fb-52a095c05533-metrics-certs\") pod \"maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv\" (UID: \"47e07161-1cfe-4075-84fb-52a095c05533\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"
Oct 30 19:04:13.454525 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.454485    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cgzn\" (UniqueName: \"kubernetes.io/projected/47e07161-1cfe-4075-84fb-52a095c05533-kube-api-access-5cgzn\") pod \"maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv\" (UID: \"47e07161-1cfe-4075-84fb-52a095c05533\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"
Oct 30 19:04:13.575210 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.575177    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:13.772762 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.772721    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:13.916090 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.916014    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47e07161-1cfe-4075-84fb-52a095c05533-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv\" (UID: \"47e07161-1cfe-4075-84fb-52a095c05533\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"
Oct 30 19:04:13.918637 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.918609    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47e07161-1cfe-4075-84fb-52a095c05533-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv\" (UID: \"47e07161-1cfe-4075-84fb-52a095c05533\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"
Oct 30 19:04:13.968116 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:13.968057    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:14.150360 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:14.150319    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"
Oct 30 19:04:14.533705 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:14.533676    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"]
Oct 30 19:04:14.535343 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:04:14.535314    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47e07161_1cfe_4075_84fb_52a095c05533.slice/crio-855671f34dbf6f8bf6305a86a2d45d981bc904e17c3a465ac041bea058bd90ed WatchSource:0}: Error finding container 855671f34dbf6f8bf6305a86a2d45d981bc904e17c3a465ac041bea058bd90ed: Status 404 returned error can't find the container with id 855671f34dbf6f8bf6305a86a2d45d981bc904e17c3a465ac041bea058bd90ed
Oct 30 19:04:14.646565 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:14.646529    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv" event={"ID":"47e07161-1cfe-4075-84fb-52a095c05533","Type":"ContainerStarted","Data":"855671f34dbf6f8bf6305a86a2d45d981bc904e17c3a465ac041bea058bd90ed"}
Oct 30 19:04:15.299629 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:15.299596    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:15.654735 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:15.654639    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv" event={"ID":"47e07161-1cfe-4075-84fb-52a095c05533","Type":"ContainerStarted","Data":"b1067935dff729362446d98e0407337c2752bdebed682fa24e9f8fcd131e437e"}
Oct 30 19:04:15.654735 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:15.654712    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"
Oct 30 19:04:15.779363 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:15.779308    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv" podStartSLOduration=1.970559615 podStartE2EDuration="2.779293307s" totalImagesPullingTime="808.733692ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:04:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:04:15.776968379 +0000 UTC m=+1776.488717072" watchObservedRunningTime="2026-10-30 19:04:15.779293307 +0000 UTC m=+1776.491042037"
Oct 30 19:04:15.945923 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:15.945851    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:16.213553 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:16.213467    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:16.437010 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:16.427754    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:16.626975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:16.625048    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:17.561827 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:17.561792    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:18.552575 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.552540    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"]
Oct 30 19:04:18.557260 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.557240    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.633366 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.633335    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"]
Oct 30 19:04:18.653992 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.653961    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fd20f13-a79c-4b16-b85c-56827c10e159-istiod-ca-cert\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.654134 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.654094    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-credential-socket\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.654184 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.654169    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mghzs\" (UniqueName: \"kubernetes.io/projected/3fd20f13-a79c-4b16-b85c-56827c10e159-kube-api-access-mghzs\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.654226 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.654212    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-envoy\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.654265 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.654240    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-podinfo\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.654392 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.654378    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-workload-certs\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.654454 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.654413    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-token\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.654454 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.654442    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-workload-socket\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.654539 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.654524    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-data\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.660140 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.660105    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-1acf9b-openshift-default-dockercfg-vw6s8\""
Oct 30 19:04:18.683815 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.683777    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:18.755334 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.755309    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-token\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.767857 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.755368    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-workload-socket\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.767857 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.755425    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-data\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.767857 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.755477    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fd20f13-a79c-4b16-b85c-56827c10e159-istiod-ca-cert\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.767857 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.755593    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-credential-socket\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.767857 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.755653    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mghzs\" (UniqueName: \"kubernetes.io/projected/3fd20f13-a79c-4b16-b85c-56827c10e159-kube-api-access-mghzs\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.767857 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.755710    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-envoy\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.767857 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.755757    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-podinfo\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.767857 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.755812    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-workload-certs\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.767857 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.756265    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-workload-certs\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.767857 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.756805    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-workload-socket\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.767857 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.757089    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-data\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.767857 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.757688    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fd20f13-a79c-4b16-b85c-56827c10e159-istiod-ca-cert\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.767857 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.757925    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-credential-socket\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.791250 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.789971    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-podinfo\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.792722 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.792697    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-envoy\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.850023 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.828929    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-token\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.850023 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.832463    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mghzs\" (UniqueName: \"kubernetes.io/projected/3fd20f13-a79c-4b16-b85c-56827c10e159-kube-api-access-mghzs\") pod \"e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") " pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.886031 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.875311    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:18.931464 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:18.929254    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:19.107019 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:19.106920    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:19.353936 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:19.353908    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"]
Oct 30 19:04:19.355257 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:04:19.355231    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fd20f13_a79c_4b16_b85c_56827c10e159.slice/crio-4445ccc30cdcc4b14bd90fada8f7075816831c3bcdb5c9e864e55df63bea6bd7 WatchSource:0}: Error finding container 4445ccc30cdcc4b14bd90fada8f7075816831c3bcdb5c9e864e55df63bea6bd7: Status 404 returned error can't find the container with id 4445ccc30cdcc4b14bd90fada8f7075816831c3bcdb5c9e864e55df63bea6bd7
Oct 30 19:04:19.357843 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:19.357790    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:04:19.357901 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:19.357854    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:04:19.357901 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:19.357884    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:04:19.753467 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:19.753370    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp" event={"ID":"3fd20f13-a79c-4b16-b85c-56827c10e159","Type":"ContainerStarted","Data":"454a8da06583e19d770cffddac86965d58f09eddd5ccf031fcf3cb3f7ee0c243"}
Oct 30 19:04:19.753467 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:19.753429    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp" event={"ID":"3fd20f13-a79c-4b16-b85c-56827c10e159","Type":"ContainerStarted","Data":"4445ccc30cdcc4b14bd90fada8f7075816831c3bcdb5c9e864e55df63bea6bd7"}
Oct 30 19:04:19.875765 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:19.875709    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp" podStartSLOduration=1.8756911939999998 podStartE2EDuration="1.875691194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:04:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:04:19.873129389 +0000 UTC m=+1780.584878086" watchObservedRunningTime="2026-10-30 19:04:19.875691194 +0000 UTC m=+1780.587439892"
Oct 30 19:04:19.876479 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:19.876459    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:20.505359 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:20.505314    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:20.682346 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:20.682308    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:20.871314 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:20.871258    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:20.881849 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:20.881824    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:21.703477 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:21.703444    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"
Oct 30 19:04:21.767657 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:21.767624    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:21.768589 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:21.768567    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:04:23.764558 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:23.764517    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:23.994989 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:23.994917    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:24.191690 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:24.191649    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:24.396624 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:24.396357    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj"]
Oct 30 19:04:24.484277 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:24.484056    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:24.487748 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:24.485132    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj"]
Oct 30 19:04:24.487748 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:24.487481    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj"
Oct 30 19:04:24.494301 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:24.494264    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v55b4\" (UniqueName: \"kubernetes.io/projected/92af8465-a779-4904-b08c-aea834e86824-kube-api-access-v55b4\") pod \"payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj\" (UID: \"92af8465-a779-4904-b08c-aea834e86824\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj"
Oct 30 19:04:24.494630 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:24.494599    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92af8465-a779-4904-b08c-aea834e86824-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj\" (UID: \"92af8465-a779-4904-b08c-aea834e86824\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj"
Oct 30 19:04:24.540369 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:24.540341    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-fee9f0\""
Oct 30 19:04:24.589781 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:24.589758    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-fee9f0-dockercfg-j29k4\""
Oct 30 19:04:24.595844 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:24.595816    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92af8465-a779-4904-b08c-aea834e86824-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj\" (UID: \"92af8465-a779-4904-b08c-aea834e86824\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj"
Oct 30 19:04:24.595928 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:24.595897    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v55b4\" (UniqueName: \"kubernetes.io/projected/92af8465-a779-4904-b08c-aea834e86824-kube-api-access-v55b4\") pod \"payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj\" (UID: \"92af8465-a779-4904-b08c-aea834e86824\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj"
Oct 30 19:04:24.596665 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:24.596647    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92af8465-a779-4904-b08c-aea834e86824-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj\" (UID: \"92af8465-a779-4904-b08c-aea834e86824\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj"
Oct 30 19:04:24.657560 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:24.657524    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v55b4\" (UniqueName: \"kubernetes.io/projected/92af8465-a779-4904-b08c-aea834e86824-kube-api-access-v55b4\") pod \"payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj\" (UID: \"92af8465-a779-4904-b08c-aea834e86824\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj"
Oct 30 19:04:24.809739 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:24.809713    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj"
Oct 30 19:04:25.197051 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:04:25.197018    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92af8465_a779_4904_b08c_aea834e86824.slice/crio-db4e7ee21ac6a270c12b49cb17f65af9989f4aebf8a9afdd5259f3af05e38c87 WatchSource:0}: Error finding container db4e7ee21ac6a270c12b49cb17f65af9989f4aebf8a9afdd5259f3af05e38c87: Status 404 returned error can't find the container with id db4e7ee21ac6a270c12b49cb17f65af9989f4aebf8a9afdd5259f3af05e38c87
Oct 30 19:04:25.198177 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:25.198151    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj"]
Oct 30 19:04:25.803107 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:25.803053    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj" event={"ID":"92af8465-a779-4904-b08c-aea834e86824","Type":"ContainerStarted","Data":"7d4209daed8cc811636e022eb81b83f826f9110d20925323a49d00b0a1b1bbe7"}
Oct 30 19:04:25.803308 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:25.803123    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj" event={"ID":"92af8465-a779-4904-b08c-aea834e86824","Type":"ContainerStarted","Data":"db4e7ee21ac6a270c12b49cb17f65af9989f4aebf8a9afdd5259f3af05e38c87"}
Oct 30 19:04:25.803308 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:25.803266    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj"
Oct 30 19:04:25.921866 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:25.921817    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj" podStartSLOduration=1.921802892 podStartE2EDuration="1.921802892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:04:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:04:25.91909232 +0000 UTC m=+1786.630841020" watchObservedRunningTime="2026-10-30 19:04:25.921802892 +0000 UTC m=+1786.633551589"
Oct 30 19:04:28.770583 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:28.770552    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:29.325971 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:29.317337    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf"]
Oct 30 19:04:29.353974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:29.346069    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf"
Oct 30 19:04:29.395974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:29.395397    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf"]
Oct 30 19:04:29.400468 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:29.399686    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-229mr\" (UniqueName: \"kubernetes.io/projected/5fca5c67-64a5-44f6-9fb1-59d47874e39f-kube-api-access-229mr\") pod \"payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf\" (UID: \"5fca5c67-64a5-44f6-9fb1-59d47874e39f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf"
Oct 30 19:04:29.413352 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:29.413318    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fca5c67-64a5-44f6-9fb1-59d47874e39f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf\" (UID: \"5fca5c67-64a5-44f6-9fb1-59d47874e39f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf"
Oct 30 19:04:29.439972 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:29.437211    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-1acf9b\""
Oct 30 19:04:29.439972 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:29.437419    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-1acf9b-dockercfg-ltlhm\""
Oct 30 19:04:29.522219 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:29.522186    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-229mr\" (UniqueName: \"kubernetes.io/projected/5fca5c67-64a5-44f6-9fb1-59d47874e39f-kube-api-access-229mr\") pod \"payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf\" (UID: \"5fca5c67-64a5-44f6-9fb1-59d47874e39f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf"
Oct 30 19:04:29.522401 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:29.522276    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fca5c67-64a5-44f6-9fb1-59d47874e39f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf\" (UID: \"5fca5c67-64a5-44f6-9fb1-59d47874e39f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf"
Oct 30 19:04:29.523071 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:29.523045    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fca5c67-64a5-44f6-9fb1-59d47874e39f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf\" (UID: \"5fca5c67-64a5-44f6-9fb1-59d47874e39f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf"
Oct 30 19:04:29.583542 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:29.583479    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-229mr\" (UniqueName: \"kubernetes.io/projected/5fca5c67-64a5-44f6-9fb1-59d47874e39f-kube-api-access-229mr\") pod \"payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf\" (UID: \"5fca5c67-64a5-44f6-9fb1-59d47874e39f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf"
Oct 30 19:04:29.740978 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:29.736212    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf"
Oct 30 19:04:30.141371 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:30.141341    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf"]
Oct 30 19:04:30.141893 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:04:30.141868    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fca5c67_64a5_44f6_9fb1_59d47874e39f.slice/crio-0b19fb78f3c8f80a586f8a17800c03d78bf1a7a1f908463748a0b5b3162b8b46 WatchSource:0}: Error finding container 0b19fb78f3c8f80a586f8a17800c03d78bf1a7a1f908463748a0b5b3162b8b46: Status 404 returned error can't find the container with id 0b19fb78f3c8f80a586f8a17800c03d78bf1a7a1f908463748a0b5b3162b8b46
Oct 30 19:04:30.856576 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:30.856535    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf" event={"ID":"5fca5c67-64a5-44f6-9fb1-59d47874e39f","Type":"ContainerStarted","Data":"5de4ac45090f024ec81cf7c82946009ef96e561e6ac1cbeb69fe3302c47c0680"}
Oct 30 19:04:30.856753 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:30.856599    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf" event={"ID":"5fca5c67-64a5-44f6-9fb1-59d47874e39f","Type":"ContainerStarted","Data":"0b19fb78f3c8f80a586f8a17800c03d78bf1a7a1f908463748a0b5b3162b8b46"}
Oct 30 19:04:30.856793 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:30.856768    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf"
Oct 30 19:04:30.974978 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:30.974900    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf" podStartSLOduration=1.97488621 podStartE2EDuration="1.97488621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:04:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:04:30.971866895 +0000 UTC m=+1791.683615591" watchObservedRunningTime="2026-10-30 19:04:30.97488621 +0000 UTC m=+1791.686634906"
Oct 30 19:04:32.119409 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:32.119362    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:34.828256 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:34.823219    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:36.813142 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:36.813118    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj"
Oct 30 19:04:38.204366 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:04:38.204329    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:04:39.935298 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:39.935252    5820 scope.go:122] "RemoveContainer" containerID="f6f79e29a8155849464356ea5327e87a2acd0e7c4ab4615086112f7cafda5e5d"
Oct 30 19:04:40.083635 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:40.083594    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:41.866602 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:41.866570    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf"
Oct 30 19:04:46.566127 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:46.566089    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:50.548887 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:50.548855    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"]
Oct 30 19:04:50.549349 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:50.549069    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m" podUID="5b3120c6-159a-401d-b8ea-bc03ebbd056e" containerName="maas-api" containerID="cri-o://c145265cba191e358f914a6f5c550815119b3b2ccd268571260c791a8fa249a1" gracePeriod=30
Oct 30 19:04:50.725972 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:50.722701    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:50.899616 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:50.899591    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"
Oct 30 19:04:51.029717 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:51.029074    5820 generic.go:356] "Generic (PLEG): container finished" podID="5b3120c6-159a-401d-b8ea-bc03ebbd056e" containerID="c145265cba191e358f914a6f5c550815119b3b2ccd268571260c791a8fa249a1" exitCode=0
Oct 30 19:04:51.029717 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:51.029306    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m" event={"ID":"5b3120c6-159a-401d-b8ea-bc03ebbd056e","Type":"ContainerDied","Data":"c145265cba191e358f914a6f5c550815119b3b2ccd268571260c791a8fa249a1"}
Oct 30 19:04:51.029717 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:51.029355    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m" event={"ID":"5b3120c6-159a-401d-b8ea-bc03ebbd056e","Type":"ContainerDied","Data":"5b4aa03e758b72d49e7acf9b4533fea0d4d5ab8b3c91944a222281191c8286e9"}
Oct 30 19:04:51.029717 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:51.029386    5820 scope.go:122] "RemoveContainer" containerID="c145265cba191e358f914a6f5c550815119b3b2ccd268571260c791a8fa249a1"
Oct 30 19:04:51.029717 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:51.029540    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"
Oct 30 19:04:51.047301 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:51.047158    5820 scope.go:122] "RemoveContainer" containerID="c145265cba191e358f914a6f5c550815119b3b2ccd268571260c791a8fa249a1"
Oct 30 19:04:51.047812 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:04:51.047734    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c145265cba191e358f914a6f5c550815119b3b2ccd268571260c791a8fa249a1\": container with ID starting with c145265cba191e358f914a6f5c550815119b3b2ccd268571260c791a8fa249a1 not found: ID does not exist" containerID="c145265cba191e358f914a6f5c550815119b3b2ccd268571260c791a8fa249a1"
Oct 30 19:04:51.047812 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:51.047779    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c145265cba191e358f914a6f5c550815119b3b2ccd268571260c791a8fa249a1"} err="failed to get container status \"c145265cba191e358f914a6f5c550815119b3b2ccd268571260c791a8fa249a1\": rpc error: code = NotFound desc = could not find container \"c145265cba191e358f914a6f5c550815119b3b2ccd268571260c791a8fa249a1\": container with ID starting with c145265cba191e358f914a6f5c550815119b3b2ccd268571260c791a8fa249a1 not found: ID does not exist"
Oct 30 19:04:51.070975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:51.069475    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5b3120c6-159a-401d-b8ea-bc03ebbd056e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b3120c6-159a-401d-b8ea-bc03ebbd056e-metrics-certs\") pod \"5b3120c6-159a-401d-b8ea-bc03ebbd056e\" (UID: \"5b3120c6-159a-401d-b8ea-bc03ebbd056e\") "
Oct 30 19:04:51.070975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:51.069558    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5b3120c6-159a-401d-b8ea-bc03ebbd056e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b3120c6-159a-401d-b8ea-bc03ebbd056e-openshift-service-ca\") pod \"5b3120c6-159a-401d-b8ea-bc03ebbd056e\" (UID: \"5b3120c6-159a-401d-b8ea-bc03ebbd056e\") "
Oct 30 19:04:51.070975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:51.069681    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b3120c6-159a-401d-b8ea-bc03ebbd056e-kube-api-access-gnb2m\" (UniqueName: \"kubernetes.io/projected/5b3120c6-159a-401d-b8ea-bc03ebbd056e-kube-api-access-gnb2m\") pod \"5b3120c6-159a-401d-b8ea-bc03ebbd056e\" (UID: \"5b3120c6-159a-401d-b8ea-bc03ebbd056e\") "
Oct 30 19:04:51.070975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:51.069843    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5b3120c6-159a-401d-b8ea-bc03ebbd056e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b3120c6-159a-401d-b8ea-bc03ebbd056e-maas-api-tls\") pod \"5b3120c6-159a-401d-b8ea-bc03ebbd056e\" (UID: \"5b3120c6-159a-401d-b8ea-bc03ebbd056e\") "
Oct 30 19:04:51.088858 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:51.088812    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b3120c6-159a-401d-b8ea-bc03ebbd056e-openshift-service-ca" pod "5b3120c6-159a-401d-b8ea-bc03ebbd056e" (UID: "5b3120c6-159a-401d-b8ea-bc03ebbd056e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:04:51.119775 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:51.119706    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b3120c6-159a-401d-b8ea-bc03ebbd056e-metrics-certs" pod "5b3120c6-159a-401d-b8ea-bc03ebbd056e" (UID: "5b3120c6-159a-401d-b8ea-bc03ebbd056e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:04:51.120382 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:51.119750    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b3120c6-159a-401d-b8ea-bc03ebbd056e-kube-api-access-gnb2m" pod "5b3120c6-159a-401d-b8ea-bc03ebbd056e" (UID: "5b3120c6-159a-401d-b8ea-bc03ebbd056e"). InnerVolumeSpecName "kube-api-access-gnb2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:04:51.120532 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:51.119816    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b3120c6-159a-401d-b8ea-bc03ebbd056e-maas-api-tls" pod "5b3120c6-159a-401d-b8ea-bc03ebbd056e" (UID: "5b3120c6-159a-401d-b8ea-bc03ebbd056e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:04:51.171727 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:51.171697    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b3120c6-159a-401d-b8ea-bc03ebbd056e-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:04:51.171727 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:51.171725    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b3120c6-159a-401d-b8ea-bc03ebbd056e-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:04:51.171927 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:51.171734    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnb2m\" (UniqueName: \"kubernetes.io/projected/5b3120c6-159a-401d-b8ea-bc03ebbd056e-kube-api-access-gnb2m\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:04:51.171927 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:51.171746    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b3120c6-159a-401d-b8ea-bc03ebbd056e-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:04:51.455632 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:51.455559    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"]
Oct 30 19:04:51.459996 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:51.459942    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4ebcd2-68b6489df6-nm86m"]
Oct 30 19:04:52.039611 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:52.039524    5820 generic.go:356] "Generic (PLEG): container finished" podID="8697cc5c-98e3-418b-9d0a-fd6930ce317e" containerID="e612c5d221069d4238d52d60e6d194e6fa8a4d138763330d9041265d56756db2" exitCode=0
Oct 30 19:04:52.039611 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:52.039576    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n" event={"ID":"8697cc5c-98e3-418b-9d0a-fd6930ce317e","Type":"ContainerDied","Data":"e612c5d221069d4238d52d60e6d194e6fa8a4d138763330d9041265d56756db2"}
Oct 30 19:04:52.040100 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:52.039924    5820 scope.go:122] "RemoveContainer" containerID="e612c5d221069d4238d52d60e6d194e6fa8a4d138763330d9041265d56756db2"
Oct 30 19:04:52.041498 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:52.041468    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:04:52.041569 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:52.041537    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:04:52.041610 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:52.041573    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:04:52.388762 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:52.388722    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:52.772379 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:52.770933    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:53.055790 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:53.055734    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n" event={"ID":"8697cc5c-98e3-418b-9d0a-fd6930ce317e","Type":"ContainerStarted","Data":"84051cc564143f2a0f9f0ee28eda83122ad8a7dfd4c88712c9b5c4846fbfbc47"}
Oct 30 19:04:53.460026 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:53.459542    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b3120c6-159a-401d-b8ea-bc03ebbd056e" path="/var/lib/kubelet/pods/5b3120c6-159a-401d-b8ea-bc03ebbd056e/volumes"
Oct 30 19:04:53.480472 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:53.480434    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:53.514598 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:53.514575    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:04:53.516058 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:53.516035    5820 patch_prober.go:29] interesting pod/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 30 19:04:53.516166 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:53.516080    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n" podUID="8697cc5c-98e3-418b-9d0a-fd6930ce317e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 30 19:04:54.515592 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:54.515556    5820 patch_prober.go:29] interesting pod/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 30 19:04:54.516012 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:54.515607    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n" podUID="8697cc5c-98e3-418b-9d0a-fd6930ce317e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 30 19:04:55.515208 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:55.515177    5820 patch_prober.go:29] interesting pod/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 30 19:04:55.515398 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:55.515227    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n" podUID="8697cc5c-98e3-418b-9d0a-fd6930ce317e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 30 19:04:56.262224 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:56.262187    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:04:56.515550 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:56.515467    5820 patch_prober.go:29] interesting pod/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 30 19:04:56.515550 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:56.515524    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n" podUID="8697cc5c-98e3-418b-9d0a-fd6930ce317e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 30 19:04:56.774021 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:56.773509    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:57.515330 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:57.515292    5820 patch_prober.go:29] interesting pod/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 30 19:04:57.515759 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:57.515352    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n" podUID="8697cc5c-98e3-418b-9d0a-fd6930ce317e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 30 19:04:57.845973 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:57.845004    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:58.514676 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:58.514636    5820 patch_prober.go:29] interesting pod/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 30 19:04:58.514854 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:58.514721    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n" podUID="8697cc5c-98e3-418b-9d0a-fd6930ce317e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 30 19:04:59.322048 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:59.322012    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:59.518699 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:59.518668    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:04:59.519707 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:04:59.519687    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:05:00.286560 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:00.286498    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:01.079231 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.079195    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:01.199393 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.199361    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"]
Oct 30 19:05:01.200070 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.200051    5820 cpu_manager.go:414] "containerMap: removing container" podUID="5b3120c6-159a-401d-b8ea-bc03ebbd056e" containerName="maas-api"
Oct 30 19:05:01.200070 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.200069    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b3120c6-159a-401d-b8ea-bc03ebbd056e" containerName="maas-api"
Oct 30 19:05:01.200223 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.200195    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b3120c6-159a-401d-b8ea-bc03ebbd056e" containerName="maas-api"
Oct 30 19:05:01.203912 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.203896    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.279203 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.279169    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-e67448-openshift-default-dockercfg-sjhqn\""
Oct 30 19:05:01.297650 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.297615    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-credential-socket\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.297855 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.297665    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-envoy\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.297855 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.297810    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8kk8\" (UniqueName: \"kubernetes.io/projected/cda46604-6d15-4ee9-823f-cb5cb70f4c51-kube-api-access-m8kk8\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.297936 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.297852    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istiod-ca-cert\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.298028 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.297933    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-token\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.298133 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.298112    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-podinfo\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.298180 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.298166    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-workload-socket\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.298215 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.298201    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-data\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.298284 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.298270    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-workload-certs\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.315028 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.314994    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"]
Oct 30 19:05:01.334038 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.333931    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"]
Oct 30 19:05:01.346385 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.345546    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs" podUID="9d11b7b9-1021-4bd9-8f74-ea0e091a2a82" containerName="istio-proxy" containerID="cri-o://343855aae3287a8e82cc8c3735eac6ad8af7e37d01a80395f2a6192735048f8d" gracePeriod=30
Oct 30 19:05:01.395036 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.395008    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"]
Oct 30 19:05:01.395396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.395374    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n" podUID="e3353c6a-b313-45af-a000-d826722d1416" containerName="istio-proxy" containerID="cri-o://65d21ec774f53dcabc16d8eaca22aff792cfce7737d28fbe8ed8bc295e023da2" gracePeriod=30
Oct 30 19:05:01.401526 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.401493    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-podinfo\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.401782 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.401760    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-workload-socket\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.402638 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.402613    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-data\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.402865 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.402840    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"]
Oct 30 19:05:01.403201 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.403176    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8" podUID="8d39a7cd-562c-49cc-9ce2-63e30d53eae0" containerName="istio-proxy" containerID="cri-o://47c250b9e9e1786910a967902260af6afcbcb6aec9f0ac84bf401fe6682f6ccd" gracePeriod=30
Oct 30 19:05:01.403434 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.402841    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-workload-certs\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.403632 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.403613    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-credential-socket\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.403748 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.403733    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-envoy\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.403892 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.403876    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8kk8\" (UniqueName: \"kubernetes.io/projected/cda46604-6d15-4ee9-823f-cb5cb70f4c51-kube-api-access-m8kk8\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.404023 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.404009    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istiod-ca-cert\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.404189 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.404173    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-token\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.404753 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.403247    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-workload-certs\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.405555 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.405340    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-workload-socket\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.405627 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.405615    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-data\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.406677 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.406635    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"]
Oct 30 19:05:01.412629 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.406832    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms" podUID="5dece09c-bff0-403f-a991-b204dc1f0f4c" containerName="istio-proxy" containerID="cri-o://1dc2bec8cabd6e2680df9b1827d1820604a1ac66ea0538d17ced37930401dd37" gracePeriod=30
Oct 30 19:05:01.412629 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.407406    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-credential-socket\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.412629 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.411085    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istiod-ca-cert\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.412908 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.412719    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-podinfo\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.414406 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.414361    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-envoy\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.419076 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.417460    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:01.502353 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.501813    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8kk8\" (UniqueName: \"kubernetes.io/projected/cda46604-6d15-4ee9-823f-cb5cb70f4c51-kube-api-access-m8kk8\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.502353 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.502302    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-token\") pod \"e2e-api-b-e67448-openshift-default-568d47b899-v6w98\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") " pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.538985 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.523636    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:01.686070 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.683572    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:05:01.686070 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.683640    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs" podUID="9d11b7b9-1021-4bd9-8f74-ea0e091a2a82" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:05:01.704714 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.704679    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:05:01.704851 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.704730    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms" podUID="5dece09c-bff0-403f-a991-b204dc1f0f4c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:05:01.726089 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:01.725968    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:02.089238 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:02.089212    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"]
Oct 30 19:05:02.090682 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:05:02.090656    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcda46604_6d15_4ee9_823f_cb5cb70f4c51.slice/crio-c52849627446ff1fd87f8479c13a2f510988415ef5ab3d12f30635ef1d6c0672 WatchSource:0}: Error finding container c52849627446ff1fd87f8479c13a2f510988415ef5ab3d12f30635ef1d6c0672: Status 404 returned error can't find the container with id c52849627446ff1fd87f8479c13a2f510988415ef5ab3d12f30635ef1d6c0672
Oct 30 19:05:02.093309 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:02.093277    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:05:02.093392 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:02.093342    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:05:02.093392 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:02.093375    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:05:02.157972 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:02.156264    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:05:02.157972 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:02.156333    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n" podUID="e3353c6a-b313-45af-a000-d826722d1416" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:05:02.171396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:02.171365    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98" event={"ID":"cda46604-6d15-4ee9-823f-cb5cb70f4c51","Type":"ContainerStarted","Data":"c52849627446ff1fd87f8479c13a2f510988415ef5ab3d12f30635ef1d6c0672"}
Oct 30 19:05:02.470472 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:02.470379    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:03.226024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:03.208056    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98" event={"ID":"cda46604-6d15-4ee9-823f-cb5cb70f4c51","Type":"ContainerStarted","Data":"488e8b395f1d7baa8912b4bda18e4ba773af44820105b23c6943603e130aa9df"}
Oct 30 19:05:03.307387 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:03.307343    5820 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:05:03.307576 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:03.307443    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8" podUID="8d39a7cd-562c-49cc-9ce2-63e30d53eae0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:05:03.367976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:03.367756    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98" podStartSLOduration=2.367662447 podStartE2EDuration="2.367662447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:03.364501594 +0000 UTC m=+1824.076250305" watchObservedRunningTime="2026-10-30 19:05:03.367662447 +0000 UTC m=+1824.079411145"
Oct 30 19:05:03.412907 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:03.412865    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:03.534073 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:03.533995    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:03.541931 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:03.541902    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:03.718120 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:03.718083    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:04.215677 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:04.215634    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:04.216725 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:04.216701    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:05:04.231239 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:04.231208    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:04.498168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:04.498039    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:05.485705 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:05.485667    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:05.851052 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:05.851009    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:06.417916 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.417878    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:06.663645 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.663457    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:06.744991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.744924    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:05:06.749684 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.749658    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:05:06.768862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.767780    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3353c6a-b313-45af-a000-d826722d1416-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3353c6a-b313-45af-a000-d826722d1416-istiod-ca-cert\") pod \"e3353c6a-b313-45af-a000-d826722d1416\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") "
Oct 30 19:05:06.768862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.767841    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-kube-api-access-wcnmb\" (UniqueName: \"kubernetes.io/projected/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-kube-api-access-wcnmb\") pod \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") "
Oct 30 19:05:06.768862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.767870    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3353c6a-b313-45af-a000-d826722d1416-istio-token\" (UniqueName: \"kubernetes.io/projected/e3353c6a-b313-45af-a000-d826722d1416-istio-token\") pod \"e3353c6a-b313-45af-a000-d826722d1416\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") "
Oct 30 19:05:06.768862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.767935    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-workload-socket\") pod \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") "
Oct 30 19:05:06.768862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.768008    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-data\") pod \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") "
Oct 30 19:05:06.768862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.768035    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-credential-socket\") pod \"e3353c6a-b313-45af-a000-d826722d1416\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") "
Oct 30 19:05:06.768862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.768066    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-istio-envoy\") pod \"e3353c6a-b313-45af-a000-d826722d1416\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") "
Oct 30 19:05:06.768862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.768124    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-credential-socket\") pod \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") "
Oct 30 19:05:06.768862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.768199    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-envoy\") pod \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") "
Oct 30 19:05:06.768862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.768252    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-istio-data\") pod \"e3353c6a-b313-45af-a000-d826722d1416\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") "
Oct 30 19:05:06.768862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.768350    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-token\" (UniqueName: \"kubernetes.io/projected/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-token\") pod \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") "
Oct 30 19:05:06.768862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.768436    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-workload-certs\") pod \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") "
Oct 30 19:05:06.768862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.768492    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-podinfo\") pod \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") "
Oct 30 19:05:06.768862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.768555    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-workload-socket\") pod \"e3353c6a-b313-45af-a000-d826722d1416\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") "
Oct 30 19:05:06.768862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.768597    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-workload-certs\") pod \"e3353c6a-b313-45af-a000-d826722d1416\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") "
Oct 30 19:05:06.768862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.768692    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istiod-ca-cert\") pod \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\" (UID: \"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82\") "
Oct 30 19:05:06.768862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.768719    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e3353c6a-b313-45af-a000-d826722d1416-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3353c6a-b313-45af-a000-d826722d1416-istio-podinfo\") pod \"e3353c6a-b313-45af-a000-d826722d1416\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") "
Oct 30 19:05:06.768862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.768756    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3353c6a-b313-45af-a000-d826722d1416-kube-api-access-sntvn\" (UniqueName: \"kubernetes.io/projected/e3353c6a-b313-45af-a000-d826722d1416-kube-api-access-sntvn\") pod \"e3353c6a-b313-45af-a000-d826722d1416\" (UID: \"e3353c6a-b313-45af-a000-d826722d1416\") "
Oct 30 19:05:06.797356 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.771497    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-workload-socket" pod "9d11b7b9-1021-4bd9-8f74-ea0e091a2a82" (UID: "9d11b7b9-1021-4bd9-8f74-ea0e091a2a82"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.797356 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.771717    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-workload-certs" pod "9d11b7b9-1021-4bd9-8f74-ea0e091a2a82" (UID: "9d11b7b9-1021-4bd9-8f74-ea0e091a2a82"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.797356 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.771927    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-workload-certs" pod "e3353c6a-b313-45af-a000-d826722d1416" (UID: "e3353c6a-b313-45af-a000-d826722d1416"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.797356 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.772134    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-workload-socket" pod "e3353c6a-b313-45af-a000-d826722d1416" (UID: "e3353c6a-b313-45af-a000-d826722d1416"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.797356 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.772181    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-istio-data" pod "e3353c6a-b313-45af-a000-d826722d1416" (UID: "e3353c6a-b313-45af-a000-d826722d1416"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.797356 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.772306    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3353c6a-b313-45af-a000-d826722d1416-istiod-ca-cert" pod "e3353c6a-b313-45af-a000-d826722d1416" (UID: "e3353c6a-b313-45af-a000-d826722d1416"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:05:06.797356 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.775546    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istiod-ca-cert" pod "9d11b7b9-1021-4bd9-8f74-ea0e091a2a82" (UID: "9d11b7b9-1021-4bd9-8f74-ea0e091a2a82"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:05:06.797356 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.776517    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-data" pod "9d11b7b9-1021-4bd9-8f74-ea0e091a2a82" (UID: "9d11b7b9-1021-4bd9-8f74-ea0e091a2a82"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.797356 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.776770    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-credential-socket" pod "e3353c6a-b313-45af-a000-d826722d1416" (UID: "e3353c6a-b313-45af-a000-d826722d1416"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.797356 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.778224    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-credential-socket" pod "9d11b7b9-1021-4bd9-8f74-ea0e091a2a82" (UID: "9d11b7b9-1021-4bd9-8f74-ea0e091a2a82"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.797356 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.780192    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-istio-envoy" pod "e3353c6a-b313-45af-a000-d826722d1416" (UID: "e3353c6a-b313-45af-a000-d826722d1416"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.797356 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.783494    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e3353c6a-b313-45af-a000-d826722d1416-istio-podinfo" pod "e3353c6a-b313-45af-a000-d826722d1416" (UID: "e3353c6a-b313-45af-a000-d826722d1416"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:05:06.797356 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.796083    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-podinfo" pod "9d11b7b9-1021-4bd9-8f74-ea0e091a2a82" (UID: "9d11b7b9-1021-4bd9-8f74-ea0e091a2a82"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:05:06.797356 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.796176    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-token" pod "9d11b7b9-1021-4bd9-8f74-ea0e091a2a82" (UID: "9d11b7b9-1021-4bd9-8f74-ea0e091a2a82"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:06.797356 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.796647    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3353c6a-b313-45af-a000-d826722d1416-istio-token" pod "e3353c6a-b313-45af-a000-d826722d1416" (UID: "e3353c6a-b313-45af-a000-d826722d1416"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:06.798347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.798315    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3353c6a-b313-45af-a000-d826722d1416-kube-api-access-sntvn" pod "e3353c6a-b313-45af-a000-d826722d1416" (UID: "e3353c6a-b313-45af-a000-d826722d1416"). InnerVolumeSpecName "kube-api-access-sntvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:06.799399 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.799373    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-envoy" pod "9d11b7b9-1021-4bd9-8f74-ea0e091a2a82" (UID: "9d11b7b9-1021-4bd9-8f74-ea0e091a2a82"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.802311 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.802285    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-kube-api-access-wcnmb" pod "9d11b7b9-1021-4bd9-8f74-ea0e091a2a82" (UID: "9d11b7b9-1021-4bd9-8f74-ea0e091a2a82"). InnerVolumeSpecName "kube-api-access-wcnmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:06.812105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.812087    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:05:06.828086 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.828068    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 19:05:06.870496 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.870441    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5dece09c-bff0-403f-a991-b204dc1f0f4c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dece09c-bff0-403f-a991-b204dc1f0f4c-istiod-ca-cert\") pod \"5dece09c-bff0-403f-a991-b204dc1f0f4c\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") "
Oct 30 19:05:06.870496 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.870488    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-workload-socket\") pod \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") "
Oct 30 19:05:06.870725 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.870534    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-envoy\") pod \"5dece09c-bff0-403f-a991-b204dc1f0f4c\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") "
Oct 30 19:05:06.870725 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.870580    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-workload-certs\") pod \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") "
Oct 30 19:05:06.870725 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.870681    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-data\") pod \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") "
Oct 30 19:05:06.870725 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.870714    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istiod-ca-cert\") pod \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") "
Oct 30 19:05:06.872168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.870745    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-envoy\") pod \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") "
Oct 30 19:05:06.872168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.870765    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-workload-socket" pod "8d39a7cd-562c-49cc-9ce2-63e30d53eae0" (UID: "8d39a7cd-562c-49cc-9ce2-63e30d53eae0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.872168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.870787    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-token\" (UniqueName: \"kubernetes.io/projected/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-token\") pod \"5dece09c-bff0-403f-a991-b204dc1f0f4c\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") "
Oct 30 19:05:06.872168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.870825    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-workload-socket\") pod \"5dece09c-bff0-403f-a991-b204dc1f0f4c\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") "
Oct 30 19:05:06.872168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.870863    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-workload-certs\") pod \"5dece09c-bff0-403f-a991-b204dc1f0f4c\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") "
Oct 30 19:05:06.872168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.870895    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-podinfo\") pod \"5dece09c-bff0-403f-a991-b204dc1f0f4c\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") "
Oct 30 19:05:06.872168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.870931    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-credential-socket\") pod \"5dece09c-bff0-403f-a991-b204dc1f0f4c\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") "
Oct 30 19:05:06.872168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.871006    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-workload-certs" pod "8d39a7cd-562c-49cc-9ce2-63e30d53eae0" (UID: "8d39a7cd-562c-49cc-9ce2-63e30d53eae0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.872168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.871037    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-credential-socket\") pod \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") "
Oct 30 19:05:06.872168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.871066    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-data" pod "8d39a7cd-562c-49cc-9ce2-63e30d53eae0" (UID: "8d39a7cd-562c-49cc-9ce2-63e30d53eae0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.872168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.871072    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-kube-api-access-wvw5s\" (UniqueName: \"kubernetes.io/projected/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-kube-api-access-wvw5s\") pod \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") "
Oct 30 19:05:06.872168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.871109    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istiod-ca-cert" pod "8d39a7cd-562c-49cc-9ce2-63e30d53eae0" (UID: "8d39a7cd-562c-49cc-9ce2-63e30d53eae0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:05:06.872168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.871200    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-token\" (UniqueName: \"kubernetes.io/projected/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-token\") pod \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") "
Oct 30 19:05:06.872168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.871265    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-data\") pod \"5dece09c-bff0-403f-a991-b204dc1f0f4c\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") "
Oct 30 19:05:06.872168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.871322    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5dece09c-bff0-403f-a991-b204dc1f0f4c-kube-api-access-dx4kg\" (UniqueName: \"kubernetes.io/projected/5dece09c-bff0-403f-a991-b204dc1f0f4c-kube-api-access-dx4kg\") pod \"5dece09c-bff0-403f-a991-b204dc1f0f4c\" (UID: \"5dece09c-bff0-403f-a991-b204dc1f0f4c\") "
Oct 30 19:05:06.872168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.871471    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-podinfo\") pod \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\" (UID: \"8d39a7cd-562c-49cc-9ce2-63e30d53eae0\") "
Oct 30 19:05:06.872168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.871515    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-workload-socket" pod "5dece09c-bff0-403f-a991-b204dc1f0f4c" (UID: "5dece09c-bff0-403f-a991-b204dc1f0f4c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.872168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.871694    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-workload-certs" pod "5dece09c-bff0-403f-a991-b204dc1f0f4c" (UID: "5dece09c-bff0-403f-a991-b204dc1f0f4c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872223    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872247    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872262    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872278    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872293    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872307    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872321    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872338    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872352    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3353c6a-b313-45af-a000-d826722d1416-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872366    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sntvn\" (UniqueName: \"kubernetes.io/projected/e3353c6a-b313-45af-a000-d826722d1416-kube-api-access-sntvn\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872382    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3353c6a-b313-45af-a000-d826722d1416-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872406    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcnmb\" (UniqueName: \"kubernetes.io/projected/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-kube-api-access-wcnmb\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872419    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3353c6a-b313-45af-a000-d826722d1416-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872433    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872446    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872460    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872472    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872485    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872479    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-credential-socket" pod "8d39a7cd-562c-49cc-9ce2-63e30d53eae0" (UID: "8d39a7cd-562c-49cc-9ce2-63e30d53eae0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872498    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872538    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872558    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872572    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3353c6a-b313-45af-a000-d826722d1416-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872586    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872599    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872658    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-credential-socket" pod "5dece09c-bff0-403f-a991-b204dc1f0f4c" (UID: "5dece09c-bff0-403f-a991-b204dc1f0f4c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.873105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.872717    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-data" pod "5dece09c-bff0-403f-a991-b204dc1f0f4c" (UID: "5dece09c-bff0-403f-a991-b204dc1f0f4c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.874649 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.873723    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dece09c-bff0-403f-a991-b204dc1f0f4c-istiod-ca-cert" pod "5dece09c-bff0-403f-a991-b204dc1f0f4c" (UID: "5dece09c-bff0-403f-a991-b204dc1f0f4c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:05:06.874963 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.874926    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-kube-api-access-wvw5s" pod "8d39a7cd-562c-49cc-9ce2-63e30d53eae0" (UID: "8d39a7cd-562c-49cc-9ce2-63e30d53eae0"). InnerVolumeSpecName "kube-api-access-wvw5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:06.877019 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.876994    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-token" pod "5dece09c-bff0-403f-a991-b204dc1f0f4c" (UID: "5dece09c-bff0-403f-a991-b204dc1f0f4c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:06.877123 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.877102    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-envoy" pod "5dece09c-bff0-403f-a991-b204dc1f0f4c" (UID: "5dece09c-bff0-403f-a991-b204dc1f0f4c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.877271 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.877252    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-envoy" pod "8d39a7cd-562c-49cc-9ce2-63e30d53eae0" (UID: "8d39a7cd-562c-49cc-9ce2-63e30d53eae0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.877358 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.877328    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dece09c-bff0-403f-a991-b204dc1f0f4c-kube-api-access-dx4kg" pod "5dece09c-bff0-403f-a991-b204dc1f0f4c" (UID: "5dece09c-bff0-403f-a991-b204dc1f0f4c"). InnerVolumeSpecName "kube-api-access-dx4kg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:06.878819 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.878787    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-podinfo" pod "8d39a7cd-562c-49cc-9ce2-63e30d53eae0" (UID: "8d39a7cd-562c-49cc-9ce2-63e30d53eae0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:05:06.880481 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.880461    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-token" pod "8d39a7cd-562c-49cc-9ce2-63e30d53eae0" (UID: "8d39a7cd-562c-49cc-9ce2-63e30d53eae0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:06.881105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.881086    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-podinfo" pod "5dece09c-bff0-403f-a991-b204dc1f0f4c" (UID: "5dece09c-bff0-403f-a991-b204dc1f0f4c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:05:06.902558 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.902196    5820 patch_prober.go:29] interesting pod/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": read tcp 10.134.0.2:59652->10.134.0.109:15021: read: connection reset by peer" start-of-body=
Oct 30 19:05:06.902558 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.902456    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8" podUID="18a0d7fb-d2d5-49b6-9d2c-be64503459de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": read tcp 10.134.0.2:59652->10.134.0.109:15021: read: connection reset by peer"
Oct 30 19:05:06.973413 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.973384    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.973413 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.973411    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.973595 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.973421    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dx4kg\" (UniqueName: \"kubernetes.io/projected/5dece09c-bff0-403f-a991-b204dc1f0f4c-kube-api-access-dx4kg\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.973595 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.973430    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.973595 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.973440    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dece09c-bff0-403f-a991-b204dc1f0f4c-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.973595 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.973449    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.973595 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.973457    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.973595 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.973466    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.973595 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.973474    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dece09c-bff0-403f-a991-b204dc1f0f4c-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.973595 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.973483    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dece09c-bff0-403f-a991-b204dc1f0f4c-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.973595 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.973491    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.973595 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:06.973500    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvw5s\" (UniqueName: \"kubernetes.io/projected/8d39a7cd-562c-49cc-9ce2-63e30d53eae0-kube-api-access-wvw5s\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:07.264290 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.264211    5820 generic.go:356] "Generic (PLEG): container finished" podID="18a0d7fb-d2d5-49b6-9d2c-be64503459de" containerID="d8b91a1a2211449bc4a71de67b331434aec2c04a4e34f8a39fbb58d40438cd62" exitCode=0
Oct 30 19:05:07.264449 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.264283    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8" event={"ID":"18a0d7fb-d2d5-49b6-9d2c-be64503459de","Type":"ContainerDied","Data":"d8b91a1a2211449bc4a71de67b331434aec2c04a4e34f8a39fbb58d40438cd62"}
Oct 30 19:05:07.264725 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.264707    5820 scope.go:122] "RemoveContainer" containerID="d8b91a1a2211449bc4a71de67b331434aec2c04a4e34f8a39fbb58d40438cd62"
Oct 30 19:05:07.265670 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.265649    5820 generic.go:356] "Generic (PLEG): container finished" podID="8d39a7cd-562c-49cc-9ce2-63e30d53eae0" containerID="47c250b9e9e1786910a967902260af6afcbcb6aec9f0ac84bf401fe6682f6ccd" exitCode=0
Oct 30 19:05:07.265770 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.265689    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"
Oct 30 19:05:07.265770 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.265721    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8" event={"ID":"8d39a7cd-562c-49cc-9ce2-63e30d53eae0","Type":"ContainerDied","Data":"47c250b9e9e1786910a967902260af6afcbcb6aec9f0ac84bf401fe6682f6ccd"}
Oct 30 19:05:07.265770 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.265765    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8" event={"ID":"8d39a7cd-562c-49cc-9ce2-63e30d53eae0","Type":"ContainerDied","Data":"d72a8c5df88c95fe14a2124b3dc3e4cf5dd638da8b931f4d1d7396e1d5c14e26"}
Oct 30 19:05:07.265913 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.265795    5820 scope.go:122] "RemoveContainer" containerID="47c250b9e9e1786910a967902260af6afcbcb6aec9f0ac84bf401fe6682f6ccd"
Oct 30 19:05:07.266375 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.266347    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:05:07.266491 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.266432    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:05:07.266491 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.266482    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:05:07.267626 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.267603    5820 generic.go:356] "Generic (PLEG): container finished" podID="5dece09c-bff0-403f-a991-b204dc1f0f4c" containerID="1dc2bec8cabd6e2680df9b1827d1820604a1ac66ea0538d17ced37930401dd37" exitCode=0
Oct 30 19:05:07.267810 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.267641    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms" event={"ID":"5dece09c-bff0-403f-a991-b204dc1f0f4c","Type":"ContainerDied","Data":"1dc2bec8cabd6e2680df9b1827d1820604a1ac66ea0538d17ced37930401dd37"}
Oct 30 19:05:07.267925 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.267845    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms" event={"ID":"5dece09c-bff0-403f-a991-b204dc1f0f4c","Type":"ContainerDied","Data":"8d8293320641c82f0e85e47741587cf42a09c5d3703024fbe529afd59e8b739d"}
Oct 30 19:05:07.267925 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.267658    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"
Oct 30 19:05:07.272207 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.272188    5820 generic.go:356] "Generic (PLEG): container finished" podID="9d11b7b9-1021-4bd9-8f74-ea0e091a2a82" containerID="343855aae3287a8e82cc8c3735eac6ad8af7e37d01a80395f2a6192735048f8d" exitCode=0
Oct 30 19:05:07.272321 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.272224    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"
Oct 30 19:05:07.272321 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.272251    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs" event={"ID":"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82","Type":"ContainerDied","Data":"343855aae3287a8e82cc8c3735eac6ad8af7e37d01a80395f2a6192735048f8d"}
Oct 30 19:05:07.272321 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.272310    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs" event={"ID":"9d11b7b9-1021-4bd9-8f74-ea0e091a2a82","Type":"ContainerDied","Data":"130c792cbef1aabdb48d69be1ba013ada3c743245284a32c77f74f1da7fc0edc"}
Oct 30 19:05:07.274066 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.274044    5820 generic.go:356] "Generic (PLEG): container finished" podID="e3353c6a-b313-45af-a000-d826722d1416" containerID="65d21ec774f53dcabc16d8eaca22aff792cfce7737d28fbe8ed8bc295e023da2" exitCode=0
Oct 30 19:05:07.274184 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.274161    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"
Oct 30 19:05:07.274276 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.274164    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n" event={"ID":"e3353c6a-b313-45af-a000-d826722d1416","Type":"ContainerDied","Data":"65d21ec774f53dcabc16d8eaca22aff792cfce7737d28fbe8ed8bc295e023da2"}
Oct 30 19:05:07.274392 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.274301    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n" event={"ID":"e3353c6a-b313-45af-a000-d826722d1416","Type":"ContainerDied","Data":"e48637d610daef46f7564e8b8f3105abb27b7ceaadf6b88246ee3608623b3908"}
Oct 30 19:05:07.283248 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.283109    5820 scope.go:122] "RemoveContainer" containerID="47c250b9e9e1786910a967902260af6afcbcb6aec9f0ac84bf401fe6682f6ccd"
Oct 30 19:05:07.283428 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:05:07.283393    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47c250b9e9e1786910a967902260af6afcbcb6aec9f0ac84bf401fe6682f6ccd\": container with ID starting with 47c250b9e9e1786910a967902260af6afcbcb6aec9f0ac84bf401fe6682f6ccd not found: ID does not exist" containerID="47c250b9e9e1786910a967902260af6afcbcb6aec9f0ac84bf401fe6682f6ccd"
Oct 30 19:05:07.283519 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.283423    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47c250b9e9e1786910a967902260af6afcbcb6aec9f0ac84bf401fe6682f6ccd"} err="failed to get container status \"47c250b9e9e1786910a967902260af6afcbcb6aec9f0ac84bf401fe6682f6ccd\": rpc error: code = NotFound desc = could not find container \"47c250b9e9e1786910a967902260af6afcbcb6aec9f0ac84bf401fe6682f6ccd\": container with ID starting with 47c250b9e9e1786910a967902260af6afcbcb6aec9f0ac84bf401fe6682f6ccd not found: ID does not exist"
Oct 30 19:05:07.283519 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.283445    5820 scope.go:122] "RemoveContainer" containerID="1dc2bec8cabd6e2680df9b1827d1820604a1ac66ea0538d17ced37930401dd37"
Oct 30 19:05:07.295674 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.295652    5820 scope.go:122] "RemoveContainer" containerID="44d724d76361f69e6e6b274c0cd909be0f1d819a288bc7adc57196990cfce6f9"
Oct 30 19:05:07.313248 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.313131    5820 scope.go:122] "RemoveContainer" containerID="1dc2bec8cabd6e2680df9b1827d1820604a1ac66ea0538d17ced37930401dd37"
Oct 30 19:05:07.313622 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:05:07.313595    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1dc2bec8cabd6e2680df9b1827d1820604a1ac66ea0538d17ced37930401dd37\": container with ID starting with 1dc2bec8cabd6e2680df9b1827d1820604a1ac66ea0538d17ced37930401dd37 not found: ID does not exist" containerID="1dc2bec8cabd6e2680df9b1827d1820604a1ac66ea0538d17ced37930401dd37"
Oct 30 19:05:07.313851 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.313814    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1dc2bec8cabd6e2680df9b1827d1820604a1ac66ea0538d17ced37930401dd37"} err="failed to get container status \"1dc2bec8cabd6e2680df9b1827d1820604a1ac66ea0538d17ced37930401dd37\": rpc error: code = NotFound desc = could not find container \"1dc2bec8cabd6e2680df9b1827d1820604a1ac66ea0538d17ced37930401dd37\": container with ID starting with 1dc2bec8cabd6e2680df9b1827d1820604a1ac66ea0538d17ced37930401dd37 not found: ID does not exist"
Oct 30 19:05:07.313851 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.313852    5820 scope.go:122] "RemoveContainer" containerID="44d724d76361f69e6e6b274c0cd909be0f1d819a288bc7adc57196990cfce6f9"
Oct 30 19:05:07.314591 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:05:07.314569    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44d724d76361f69e6e6b274c0cd909be0f1d819a288bc7adc57196990cfce6f9\": container with ID starting with 44d724d76361f69e6e6b274c0cd909be0f1d819a288bc7adc57196990cfce6f9 not found: ID does not exist" containerID="44d724d76361f69e6e6b274c0cd909be0f1d819a288bc7adc57196990cfce6f9"
Oct 30 19:05:07.314698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.314597    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44d724d76361f69e6e6b274c0cd909be0f1d819a288bc7adc57196990cfce6f9"} err="failed to get container status \"44d724d76361f69e6e6b274c0cd909be0f1d819a288bc7adc57196990cfce6f9\": rpc error: code = NotFound desc = could not find container \"44d724d76361f69e6e6b274c0cd909be0f1d819a288bc7adc57196990cfce6f9\": container with ID starting with 44d724d76361f69e6e6b274c0cd909be0f1d819a288bc7adc57196990cfce6f9 not found: ID does not exist"
Oct 30 19:05:07.314698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.314612    5820 scope.go:122] "RemoveContainer" containerID="343855aae3287a8e82cc8c3735eac6ad8af7e37d01a80395f2a6192735048f8d"
Oct 30 19:05:07.334540 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.334508    5820 scope.go:122] "RemoveContainer" containerID="343855aae3287a8e82cc8c3735eac6ad8af7e37d01a80395f2a6192735048f8d"
Oct 30 19:05:07.334895 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:05:07.334873    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"343855aae3287a8e82cc8c3735eac6ad8af7e37d01a80395f2a6192735048f8d\": container with ID starting with 343855aae3287a8e82cc8c3735eac6ad8af7e37d01a80395f2a6192735048f8d not found: ID does not exist" containerID="343855aae3287a8e82cc8c3735eac6ad8af7e37d01a80395f2a6192735048f8d"
Oct 30 19:05:07.335004 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.334904    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"343855aae3287a8e82cc8c3735eac6ad8af7e37d01a80395f2a6192735048f8d"} err="failed to get container status \"343855aae3287a8e82cc8c3735eac6ad8af7e37d01a80395f2a6192735048f8d\": rpc error: code = NotFound desc = could not find container \"343855aae3287a8e82cc8c3735eac6ad8af7e37d01a80395f2a6192735048f8d\": container with ID starting with 343855aae3287a8e82cc8c3735eac6ad8af7e37d01a80395f2a6192735048f8d not found: ID does not exist"
Oct 30 19:05:07.335004 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.334922    5820 scope.go:122] "RemoveContainer" containerID="65d21ec774f53dcabc16d8eaca22aff792cfce7737d28fbe8ed8bc295e023da2"
Oct 30 19:05:07.351081 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.349741    5820 scope.go:122] "RemoveContainer" containerID="65d21ec774f53dcabc16d8eaca22aff792cfce7737d28fbe8ed8bc295e023da2"
Oct 30 19:05:07.351081 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:05:07.350147    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65d21ec774f53dcabc16d8eaca22aff792cfce7737d28fbe8ed8bc295e023da2\": container with ID starting with 65d21ec774f53dcabc16d8eaca22aff792cfce7737d28fbe8ed8bc295e023da2 not found: ID does not exist" containerID="65d21ec774f53dcabc16d8eaca22aff792cfce7737d28fbe8ed8bc295e023da2"
Oct 30 19:05:07.351081 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.350182    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65d21ec774f53dcabc16d8eaca22aff792cfce7737d28fbe8ed8bc295e023da2"} err="failed to get container status \"65d21ec774f53dcabc16d8eaca22aff792cfce7737d28fbe8ed8bc295e023da2\": rpc error: code = NotFound desc = could not find container \"65d21ec774f53dcabc16d8eaca22aff792cfce7737d28fbe8ed8bc295e023da2\": container with ID starting with 65d21ec774f53dcabc16d8eaca22aff792cfce7737d28fbe8ed8bc295e023da2 not found: ID does not exist"
Oct 30 19:05:07.620976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.614821    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"]
Oct 30 19:05:07.638974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.630676    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-hxvm8"]
Oct 30 19:05:07.668793 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.668755    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"]
Oct 30 19:05:07.679520 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.679487    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-g52ms"]
Oct 30 19:05:07.902007 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.901853    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"]
Oct 30 19:05:07.911398 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:07.911357    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-s7nvs"]
Oct 30 19:05:08.022921 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.022849    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"]
Oct 30 19:05:08.025862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.025834    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jbc2n"]
Oct 30 19:05:08.074013 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.073982    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"]
Oct 30 19:05:08.074674 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.074660    5820 cpu_manager.go:414] "containerMap: removing container" podUID="5dece09c-bff0-403f-a991-b204dc1f0f4c" containerName="istio-proxy"
Oct 30 19:05:08.074718 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.074676    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5dece09c-bff0-403f-a991-b204dc1f0f4c" containerName="istio-proxy"
Oct 30 19:05:08.074718 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.074690    5820 cpu_manager.go:414] "containerMap: removing container" podUID="5dece09c-bff0-403f-a991-b204dc1f0f4c" containerName="istio-proxy"
Oct 30 19:05:08.074718 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.074695    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5dece09c-bff0-403f-a991-b204dc1f0f4c" containerName="istio-proxy"
Oct 30 19:05:08.074718 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.074702    5820 cpu_manager.go:414] "containerMap: removing container" podUID="8d39a7cd-562c-49cc-9ce2-63e30d53eae0" containerName="istio-proxy"
Oct 30 19:05:08.074718 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.074707    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d39a7cd-562c-49cc-9ce2-63e30d53eae0" containerName="istio-proxy"
Oct 30 19:05:08.074872 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.074749    5820 cpu_manager.go:414] "containerMap: removing container" podUID="9d11b7b9-1021-4bd9-8f74-ea0e091a2a82" containerName="istio-proxy"
Oct 30 19:05:08.074872 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.074755    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d11b7b9-1021-4bd9-8f74-ea0e091a2a82" containerName="istio-proxy"
Oct 30 19:05:08.074872 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.074771    5820 cpu_manager.go:414] "containerMap: removing container" podUID="e3353c6a-b313-45af-a000-d826722d1416" containerName="istio-proxy"
Oct 30 19:05:08.074872 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.074775    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3353c6a-b313-45af-a000-d826722d1416" containerName="istio-proxy"
Oct 30 19:05:08.074872 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.074864    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="5dece09c-bff0-403f-a991-b204dc1f0f4c" containerName="istio-proxy"
Oct 30 19:05:08.074872 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.074872    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="5dece09c-bff0-403f-a991-b204dc1f0f4c" containerName="istio-proxy"
Oct 30 19:05:08.075096 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.074878    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="8d39a7cd-562c-49cc-9ce2-63e30d53eae0" containerName="istio-proxy"
Oct 30 19:05:08.075096 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.074892    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d11b7b9-1021-4bd9-8f74-ea0e091a2a82" containerName="istio-proxy"
Oct 30 19:05:08.075096 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.074898    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3353c6a-b313-45af-a000-d826722d1416" containerName="istio-proxy"
Oct 30 19:05:08.077247 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.077233    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"
Oct 30 19:05:08.129383 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.129353    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-fee9f0-serving-cert\""
Oct 30 19:05:08.148447 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.148356    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"]
Oct 30 19:05:08.190917 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.190826    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e20e1010-00a5-490a-8aaa-93afaebe7932-maas-api-tls\") pod \"maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4\" (UID: \"e20e1010-00a5-490a-8aaa-93afaebe7932\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"
Oct 30 19:05:08.190917 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.190871    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e20e1010-00a5-490a-8aaa-93afaebe7932-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4\" (UID: \"e20e1010-00a5-490a-8aaa-93afaebe7932\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"
Oct 30 19:05:08.191173 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.191018    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdpw6\" (UniqueName: \"kubernetes.io/projected/e20e1010-00a5-490a-8aaa-93afaebe7932-kube-api-access-cdpw6\") pod \"maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4\" (UID: \"e20e1010-00a5-490a-8aaa-93afaebe7932\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"
Oct 30 19:05:08.191173 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.191091    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e20e1010-00a5-490a-8aaa-93afaebe7932-metrics-certs\") pod \"maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4\" (UID: \"e20e1010-00a5-490a-8aaa-93afaebe7932\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"
Oct 30 19:05:08.209042 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:05:08.209001    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:05:08.284296 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.284257    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8" event={"ID":"18a0d7fb-d2d5-49b6-9d2c-be64503459de","Type":"ContainerStarted","Data":"15e0c5326bfcd81f90e8fd5f05080b619e43a816fe4a1fd2df796296c6a8c60e"}
Oct 30 19:05:08.292257 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.292226    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e20e1010-00a5-490a-8aaa-93afaebe7932-maas-api-tls\") pod \"maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4\" (UID: \"e20e1010-00a5-490a-8aaa-93afaebe7932\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"
Oct 30 19:05:08.292408 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.292272    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e20e1010-00a5-490a-8aaa-93afaebe7932-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4\" (UID: \"e20e1010-00a5-490a-8aaa-93afaebe7932\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"
Oct 30 19:05:08.292408 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.292321    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdpw6\" (UniqueName: \"kubernetes.io/projected/e20e1010-00a5-490a-8aaa-93afaebe7932-kube-api-access-cdpw6\") pod \"maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4\" (UID: \"e20e1010-00a5-490a-8aaa-93afaebe7932\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"
Oct 30 19:05:08.292408 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:05:08.292375    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-serving-cert: secret "maas-api-e2e-ipp-b-fee9f0-serving-cert" not found
Oct 30 19:05:08.292522 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:05:08.292451    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e20e1010-00a5-490a-8aaa-93afaebe7932-maas-api-tls podName:e20e1010-00a5-490a-8aaa-93afaebe7932 nodeName:}" failed. No retries permitted until 2026-10-30 19:05:08.792432399 +0000 UTC m=+1829.504181088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e20e1010-00a5-490a-8aaa-93afaebe7932-maas-api-tls") pod "maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4" (UID: "e20e1010-00a5-490a-8aaa-93afaebe7932") : secret "maas-api-e2e-ipp-b-fee9f0-serving-cert" not found
Oct 30 19:05:08.292522 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.292492    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e20e1010-00a5-490a-8aaa-93afaebe7932-metrics-certs\") pod \"maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4\" (UID: \"e20e1010-00a5-490a-8aaa-93afaebe7932\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"
Oct 30 19:05:08.293073 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.293052    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e20e1010-00a5-490a-8aaa-93afaebe7932-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4\" (UID: \"e20e1010-00a5-490a-8aaa-93afaebe7932\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"
Oct 30 19:05:08.295140 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.295119    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e20e1010-00a5-490a-8aaa-93afaebe7932-metrics-certs\") pod \"maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4\" (UID: \"e20e1010-00a5-490a-8aaa-93afaebe7932\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"
Oct 30 19:05:08.389936 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.389692    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdpw6\" (UniqueName: \"kubernetes.io/projected/e20e1010-00a5-490a-8aaa-93afaebe7932-kube-api-access-cdpw6\") pod \"maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4\" (UID: \"e20e1010-00a5-490a-8aaa-93afaebe7932\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"
Oct 30 19:05:08.526200 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.526111    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:08.781890 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.781803    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:08.800037 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.800008    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e20e1010-00a5-490a-8aaa-93afaebe7932-maas-api-tls\") pod \"maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4\" (UID: \"e20e1010-00a5-490a-8aaa-93afaebe7932\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"
Oct 30 19:05:08.802556 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.802535    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e20e1010-00a5-490a-8aaa-93afaebe7932-maas-api-tls\") pod \"maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4\" (UID: \"e20e1010-00a5-490a-8aaa-93afaebe7932\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"
Oct 30 19:05:08.990090 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:08.990058    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"
Oct 30 19:05:09.216574 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:09.216538    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:05:09.216753 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:09.216718    5820 patch_prober.go:29] interesting pod/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 19:05:09.216803 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:09.216764    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8" podUID="18a0d7fb-d2d5-49b6-9d2c-be64503459de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 19:05:09.405511 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:09.405476    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"]
Oct 30 19:05:09.408349 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:05:09.408313    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode20e1010_00a5_490a_8aaa_93afaebe7932.slice/crio-fa45cf6191a4c51d372974271f0dc2296ce98e5fe41b61f1b29b8ce68a1ecb48 WatchSource:0}: Error finding container fa45cf6191a4c51d372974271f0dc2296ce98e5fe41b61f1b29b8ce68a1ecb48: Status 404 returned error can't find the container with id fa45cf6191a4c51d372974271f0dc2296ce98e5fe41b61f1b29b8ce68a1ecb48
Oct 30 19:05:09.460479 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:09.460444    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5dece09c-bff0-403f-a991-b204dc1f0f4c" path="/var/lib/kubelet/pods/5dece09c-bff0-403f-a991-b204dc1f0f4c/volumes"
Oct 30 19:05:09.461289 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:09.461258    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8d39a7cd-562c-49cc-9ce2-63e30d53eae0" path="/var/lib/kubelet/pods/8d39a7cd-562c-49cc-9ce2-63e30d53eae0/volumes"
Oct 30 19:05:09.461824 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:09.461807    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9d11b7b9-1021-4bd9-8f74-ea0e091a2a82" path="/var/lib/kubelet/pods/9d11b7b9-1021-4bd9-8f74-ea0e091a2a82/volumes"
Oct 30 19:05:09.462579 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:09.462564    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3353c6a-b313-45af-a000-d826722d1416" path="/var/lib/kubelet/pods/e3353c6a-b313-45af-a000-d826722d1416/volumes"
Oct 30 19:05:09.986722 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:09.986677    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:10.216810 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:10.216774    5820 patch_prober.go:29] interesting pod/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 19:05:10.217028 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:10.216829    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8" podUID="18a0d7fb-d2d5-49b6-9d2c-be64503459de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 19:05:10.303917 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:10.303883    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4" event={"ID":"e20e1010-00a5-490a-8aaa-93afaebe7932","Type":"ContainerStarted","Data":"fa45cf6191a4c51d372974271f0dc2296ce98e5fe41b61f1b29b8ce68a1ecb48"}
Oct 30 19:05:10.660594 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:10.660553    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:10.966222 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:10.966127    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:11.216612 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:11.216490    5820 patch_prober.go:29] interesting pod/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 19:05:11.216612 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:11.216543    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8" podUID="18a0d7fb-d2d5-49b6-9d2c-be64503459de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 19:05:11.313159 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:11.313090    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4" event={"ID":"e20e1010-00a5-490a-8aaa-93afaebe7932","Type":"ContainerStarted","Data":"119c611b32b963acf4f8f965a2dfb4526d0745ef038d649615175a78978c39a0"}
Oct 30 19:05:11.313347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:11.313327    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"
Oct 30 19:05:11.502244 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:11.501166    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4" podStartSLOduration=2.6760187760000003 podStartE2EDuration="3.501144542s" totalImagesPullingTime="825.125766ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:11.47731354 +0000 UTC m=+1832.189062240" watchObservedRunningTime="2026-10-30 19:05:11.501144542 +0000 UTC m=+1832.212893241"
Oct 30 19:05:11.628774 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:11.619784    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:11.908533 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:11.908497    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:12.216271 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:12.216192    5820 patch_prober.go:29] interesting pod/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 19:05:12.216271 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:12.216254    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8" podUID="18a0d7fb-d2d5-49b6-9d2c-be64503459de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 19:05:12.691067 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:12.691022    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx"]
Oct 30 19:05:12.778803 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:12.778769    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k"]
Oct 30 19:05:13.206921 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:13.206886    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:13.216886 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:13.216855    5820 patch_prober.go:29] interesting pod/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 19:05:13.217031 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:13.216903    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8" podUID="18a0d7fb-d2d5-49b6-9d2c-be64503459de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 19:05:13.554796 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:13.554759    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:14.220697 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:14.220666    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:05:14.221127 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:14.220978    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:05:14.221778 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:14.221762    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:05:14.333081 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:14.331681    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:16.964403 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:16.964361    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:17.206436 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.206397    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs"]
Oct 30 19:05:17.209921 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.209898    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs"
Oct 30 19:05:17.266498 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.266468    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-e67448-dockercfg-qtg6h\""
Oct 30 19:05:17.270515 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.270493    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs"]
Oct 30 19:05:17.293960 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.293924    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a9b9d94-81c3-42d9-a52d-269ed196dd90-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs\" (UID: \"6a9b9d94-81c3-42d9-a52d-269ed196dd90\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs"
Oct 30 19:05:17.294153 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.294132    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn48b\" (UniqueName: \"kubernetes.io/projected/6a9b9d94-81c3-42d9-a52d-269ed196dd90-kube-api-access-vn48b\") pod \"payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs\" (UID: \"6a9b9d94-81c3-42d9-a52d-269ed196dd90\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs"
Oct 30 19:05:17.327223 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.327195    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"
Oct 30 19:05:17.358036 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.349843    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-e67448\""
Oct 30 19:05:17.395120 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.395082    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn48b\" (UniqueName: \"kubernetes.io/projected/6a9b9d94-81c3-42d9-a52d-269ed196dd90-kube-api-access-vn48b\") pod \"payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs\" (UID: \"6a9b9d94-81c3-42d9-a52d-269ed196dd90\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs"
Oct 30 19:05:17.396373 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.395497    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a9b9d94-81c3-42d9-a52d-269ed196dd90-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs\" (UID: \"6a9b9d94-81c3-42d9-a52d-269ed196dd90\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs"
Oct 30 19:05:17.397695 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.397608    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a9b9d94-81c3-42d9-a52d-269ed196dd90-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs\" (UID: \"6a9b9d94-81c3-42d9-a52d-269ed196dd90\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs"
Oct 30 19:05:17.431076 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.427651    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4"]
Oct 30 19:05:17.462086 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.462047    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:17.462421 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.462346    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4"
Oct 30 19:05:17.477173 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.476615    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4"]
Oct 30 19:05:17.513973 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.505875    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn48b\" (UniqueName: \"kubernetes.io/projected/6a9b9d94-81c3-42d9-a52d-269ed196dd90-kube-api-access-vn48b\") pod \"payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs\" (UID: \"6a9b9d94-81c3-42d9-a52d-269ed196dd90\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs"
Oct 30 19:05:17.523344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.523307    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs"
Oct 30 19:05:17.600072 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.600036    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5136fb9d-676a-47b3-8c02-a48ea1ffba82-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4\" (UID: \"5136fb9d-676a-47b3-8c02-a48ea1ffba82\") " pod="openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4"
Oct 30 19:05:17.600241 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.600220    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnhcq\" (UniqueName: \"kubernetes.io/projected/5136fb9d-676a-47b3-8c02-a48ea1ffba82-kube-api-access-lnhcq\") pod \"payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4\" (UID: \"5136fb9d-676a-47b3-8c02-a48ea1ffba82\") " pod="openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4"
Oct 30 19:05:17.691698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.691657    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx" podUID="408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe" containerName="payload-processing" containerID="cri-o://49abf79390b38852c33b2d6ebcee2377deb53f80bd940cd8bfa82ae1714b44e5" gracePeriod=25
Oct 30 19:05:17.701828 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.701800    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5136fb9d-676a-47b3-8c02-a48ea1ffba82-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4\" (UID: \"5136fb9d-676a-47b3-8c02-a48ea1ffba82\") " pod="openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4"
Oct 30 19:05:17.701932 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.701844    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnhcq\" (UniqueName: \"kubernetes.io/projected/5136fb9d-676a-47b3-8c02-a48ea1ffba82-kube-api-access-lnhcq\") pod \"payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4\" (UID: \"5136fb9d-676a-47b3-8c02-a48ea1ffba82\") " pod="openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4"
Oct 30 19:05:17.702545 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.702525    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5136fb9d-676a-47b3-8c02-a48ea1ffba82-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4\" (UID: \"5136fb9d-676a-47b3-8c02-a48ea1ffba82\") " pod="openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4"
Oct 30 19:05:17.762882 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.762828    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnhcq\" (UniqueName: \"kubernetes.io/projected/5136fb9d-676a-47b3-8c02-a48ea1ffba82-kube-api-access-lnhcq\") pod \"payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4\" (UID: \"5136fb9d-676a-47b3-8c02-a48ea1ffba82\") " pod="openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4"
Oct 30 19:05:17.779137 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.779080    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k" podUID="e4b29a58-09b8-4ef8-be55-49dcac8dc6a1" containerName="payload-pre-processing" containerID="cri-o://975214c158c7ffb3869f47fce67a66eb7807c4be139cf9eef1c9f22d5c1638da" gracePeriod=25
Oct 30 19:05:17.813742 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.813718    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4"
Oct 30 19:05:17.893991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.893548    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:17.915569 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:17.915543    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs"]
Oct 30 19:05:17.921064 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:05:17.921013    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a9b9d94_81c3_42d9_a52d_269ed196dd90.slice/crio-10dcd6615a0b0925cf0dbc9a7deed1fbe8fbfc6c528dcd737904876845adefee WatchSource:0}: Error finding container 10dcd6615a0b0925cf0dbc9a7deed1fbe8fbfc6c528dcd737904876845adefee: Status 404 returned error can't find the container with id 10dcd6615a0b0925cf0dbc9a7deed1fbe8fbfc6c528dcd737904876845adefee
Oct 30 19:05:18.097329 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.097304    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx"
Oct 30 19:05:18.107623 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.107581    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe-plugins-config-volume\") pod \"408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe\" (UID: \"408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe\") "
Oct 30 19:05:18.108240 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.108217    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe-kube-api-access-5gpbz\" (UniqueName: \"kubernetes.io/projected/408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe-kube-api-access-5gpbz\") pod \"408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe\" (UID: \"408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe\") "
Oct 30 19:05:18.108460 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.108008    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe-plugins-config-volume" pod "408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe" (UID: "408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:05:18.109274 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.109253    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:18.110724 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.110694    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe-kube-api-access-5gpbz" pod "408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe" (UID: "408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe"). InnerVolumeSpecName "kube-api-access-5gpbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:18.147280 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.147251    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k"
Oct 30 19:05:18.199413 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.199387    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4"]
Oct 30 19:05:18.201359 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:05:18.201334    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5136fb9d_676a_47b3_8c02_a48ea1ffba82.slice/crio-9030c7054201d87a30bfddbc014e90dd6f4370075ae3696b8fedd111e68b4e46 WatchSource:0}: Error finding container 9030c7054201d87a30bfddbc014e90dd6f4370075ae3696b8fedd111e68b4e46: Status 404 returned error can't find the container with id 9030c7054201d87a30bfddbc014e90dd6f4370075ae3696b8fedd111e68b4e46
Oct 30 19:05:18.209701 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.209653    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e4b29a58-09b8-4ef8-be55-49dcac8dc6a1-kube-api-access-jbwcn\" (UniqueName: \"kubernetes.io/projected/e4b29a58-09b8-4ef8-be55-49dcac8dc6a1-kube-api-access-jbwcn\") pod \"e4b29a58-09b8-4ef8-be55-49dcac8dc6a1\" (UID: \"e4b29a58-09b8-4ef8-be55-49dcac8dc6a1\") "
Oct 30 19:05:18.209999 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.209975    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e4b29a58-09b8-4ef8-be55-49dcac8dc6a1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4b29a58-09b8-4ef8-be55-49dcac8dc6a1-plugins-config-volume\") pod \"e4b29a58-09b8-4ef8-be55-49dcac8dc6a1\" (UID: \"e4b29a58-09b8-4ef8-be55-49dcac8dc6a1\") "
Oct 30 19:05:18.210411 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.210386    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4b29a58-09b8-4ef8-be55-49dcac8dc6a1-plugins-config-volume" pod "e4b29a58-09b8-4ef8-be55-49dcac8dc6a1" (UID: "e4b29a58-09b8-4ef8-be55-49dcac8dc6a1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:05:18.210769 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.210668    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4b29a58-09b8-4ef8-be55-49dcac8dc6a1-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:18.210769 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.210699    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gpbz\" (UniqueName: \"kubernetes.io/projected/408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe-kube-api-access-5gpbz\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:18.214014 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.213985    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4b29a58-09b8-4ef8-be55-49dcac8dc6a1-kube-api-access-jbwcn" pod "e4b29a58-09b8-4ef8-be55-49dcac8dc6a1" (UID: "e4b29a58-09b8-4ef8-be55-49dcac8dc6a1"). InnerVolumeSpecName "kube-api-access-jbwcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:18.311926 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.311888    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbwcn\" (UniqueName: \"kubernetes.io/projected/e4b29a58-09b8-4ef8-be55-49dcac8dc6a1-kube-api-access-jbwcn\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:18.413284 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.413212    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:18.517021 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.516900    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs" event={"ID":"6a9b9d94-81c3-42d9-a52d-269ed196dd90","Type":"ContainerStarted","Data":"64a93225c75600500a55eefbffd2340af0546daa9f0323a705d1d2e7f1c4b78f"}
Oct 30 19:05:18.517271 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.517065    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs" event={"ID":"6a9b9d94-81c3-42d9-a52d-269ed196dd90","Type":"ContainerStarted","Data":"10dcd6615a0b0925cf0dbc9a7deed1fbe8fbfc6c528dcd737904876845adefee"}
Oct 30 19:05:18.517271 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.517105    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs"
Oct 30 19:05:18.519101 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.519073    5820 generic.go:356] "Generic (PLEG): container finished" podID="e4b29a58-09b8-4ef8-be55-49dcac8dc6a1" containerID="975214c158c7ffb3869f47fce67a66eb7807c4be139cf9eef1c9f22d5c1638da" exitCode=0
Oct 30 19:05:18.519197 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.519115    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k"
Oct 30 19:05:18.519197 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.519156    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k" event={"ID":"e4b29a58-09b8-4ef8-be55-49dcac8dc6a1","Type":"ContainerDied","Data":"975214c158c7ffb3869f47fce67a66eb7807c4be139cf9eef1c9f22d5c1638da"}
Oct 30 19:05:18.519287 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.519207    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k" event={"ID":"e4b29a58-09b8-4ef8-be55-49dcac8dc6a1","Type":"ContainerDied","Data":"25092f91ec56db4ecf797783659340c34638d16727805bb2f67342e86162e84e"}
Oct 30 19:05:18.519287 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.519231    5820 scope.go:122] "RemoveContainer" containerID="975214c158c7ffb3869f47fce67a66eb7807c4be139cf9eef1c9f22d5c1638da"
Oct 30 19:05:18.520897 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.520832    5820 generic.go:356] "Generic (PLEG): container finished" podID="408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe" containerID="49abf79390b38852c33b2d6ebcee2377deb53f80bd940cd8bfa82ae1714b44e5" exitCode=0
Oct 30 19:05:18.520897 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.520890    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx"
Oct 30 19:05:18.521077 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.520908    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx" event={"ID":"408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe","Type":"ContainerDied","Data":"49abf79390b38852c33b2d6ebcee2377deb53f80bd940cd8bfa82ae1714b44e5"}
Oct 30 19:05:18.521077 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.520999    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx" event={"ID":"408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe","Type":"ContainerDied","Data":"0eadef1f993d1e496d14949a0c239ea2654c942630d3cb47dea6fd66bd0e74dc"}
Oct 30 19:05:18.522881 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.522855    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4" event={"ID":"5136fb9d-676a-47b3-8c02-a48ea1ffba82","Type":"ContainerStarted","Data":"6c4fdebd47c460febf19fb46dbe9e1e750686939964d74abdd9137bf1da0ac0c"}
Oct 30 19:05:18.523009 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.522903    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4" event={"ID":"5136fb9d-676a-47b3-8c02-a48ea1ffba82","Type":"ContainerStarted","Data":"9030c7054201d87a30bfddbc014e90dd6f4370075ae3696b8fedd111e68b4e46"}
Oct 30 19:05:18.523009 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.523000    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4"
Oct 30 19:05:18.531102 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.530980    5820 scope.go:122] "RemoveContainer" containerID="975214c158c7ffb3869f47fce67a66eb7807c4be139cf9eef1c9f22d5c1638da"
Oct 30 19:05:18.531262 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:05:18.531238    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"975214c158c7ffb3869f47fce67a66eb7807c4be139cf9eef1c9f22d5c1638da\": container with ID starting with 975214c158c7ffb3869f47fce67a66eb7807c4be139cf9eef1c9f22d5c1638da not found: ID does not exist" containerID="975214c158c7ffb3869f47fce67a66eb7807c4be139cf9eef1c9f22d5c1638da"
Oct 30 19:05:18.531358 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.531268    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"975214c158c7ffb3869f47fce67a66eb7807c4be139cf9eef1c9f22d5c1638da"} err="failed to get container status \"975214c158c7ffb3869f47fce67a66eb7807c4be139cf9eef1c9f22d5c1638da\": rpc error: code = NotFound desc = could not find container \"975214c158c7ffb3869f47fce67a66eb7807c4be139cf9eef1c9f22d5c1638da\": container with ID starting with 975214c158c7ffb3869f47fce67a66eb7807c4be139cf9eef1c9f22d5c1638da not found: ID does not exist"
Oct 30 19:05:18.531358 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.531285    5820 scope.go:122] "RemoveContainer" containerID="49abf79390b38852c33b2d6ebcee2377deb53f80bd940cd8bfa82ae1714b44e5"
Oct 30 19:05:18.540469 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.540452    5820 scope.go:122] "RemoveContainer" containerID="49abf79390b38852c33b2d6ebcee2377deb53f80bd940cd8bfa82ae1714b44e5"
Oct 30 19:05:18.540734 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:05:18.540715    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49abf79390b38852c33b2d6ebcee2377deb53f80bd940cd8bfa82ae1714b44e5\": container with ID starting with 49abf79390b38852c33b2d6ebcee2377deb53f80bd940cd8bfa82ae1714b44e5 not found: ID does not exist" containerID="49abf79390b38852c33b2d6ebcee2377deb53f80bd940cd8bfa82ae1714b44e5"
Oct 30 19:05:18.540797 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.540745    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49abf79390b38852c33b2d6ebcee2377deb53f80bd940cd8bfa82ae1714b44e5"} err="failed to get container status \"49abf79390b38852c33b2d6ebcee2377deb53f80bd940cd8bfa82ae1714b44e5\": rpc error: code = NotFound desc = could not find container \"49abf79390b38852c33b2d6ebcee2377deb53f80bd940cd8bfa82ae1714b44e5\": container with ID starting with 49abf79390b38852c33b2d6ebcee2377deb53f80bd940cd8bfa82ae1714b44e5 not found: ID does not exist"
Oct 30 19:05:18.639984 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.639925    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs" podStartSLOduration=1.639912673 podStartE2EDuration="1.639912673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:18.636656492 +0000 UTC m=+1839.348405189" watchObservedRunningTime="2026-10-30 19:05:18.639912673 +0000 UTC m=+1839.351661386"
Oct 30 19:05:18.719528 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.719495    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"]
Oct 30 19:05:18.720299 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.720281    5820 cpu_manager.go:414] "containerMap: removing container" podUID="e4b29a58-09b8-4ef8-be55-49dcac8dc6a1" containerName="payload-pre-processing"
Oct 30 19:05:18.720358 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.720301    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e4b29a58-09b8-4ef8-be55-49dcac8dc6a1" containerName="payload-pre-processing"
Oct 30 19:05:18.720358 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.720321    5820 cpu_manager.go:414] "containerMap: removing container" podUID="408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe" containerName="payload-processing"
Oct 30 19:05:18.720358 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.720326    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe" containerName="payload-processing"
Oct 30 19:05:18.720489 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.720425    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe" containerName="payload-processing"
Oct 30 19:05:18.720489 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.720443    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="e4b29a58-09b8-4ef8-be55-49dcac8dc6a1" containerName="payload-pre-processing"
Oct 30 19:05:18.723726 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.723706    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"
Oct 30 19:05:18.791072 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.789706    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-1acf9b-serving-cert\""
Oct 30 19:05:18.827591 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.827167    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-metrics-certs\") pod \"maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2\" (UID: \"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"
Oct 30 19:05:18.827591 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.827213    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr858\" (UniqueName: \"kubernetes.io/projected/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-kube-api-access-sr858\") pod \"maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2\" (UID: \"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"
Oct 30 19:05:18.827591 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.827254    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-maas-api-tls\") pod \"maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2\" (UID: \"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"
Oct 30 19:05:18.827591 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.827558    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2\" (UID: \"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"
Oct 30 19:05:18.830985 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.830917    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx"]
Oct 30 19:05:18.834576 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.834538    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a23af5-76b489ffc6-hjpqx"]
Oct 30 19:05:18.837423 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.837400    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"]
Oct 30 19:05:18.887275 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.887243    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k"]
Oct 30 19:05:18.891364 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.891339    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a23af5-8b584fb75-64g7k"]
Oct 30 19:05:18.928622 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.928586    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2\" (UID: \"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"
Oct 30 19:05:18.928790 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.928662    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-metrics-certs\") pod \"maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2\" (UID: \"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"
Oct 30 19:05:18.928790 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.928682    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr858\" (UniqueName: \"kubernetes.io/projected/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-kube-api-access-sr858\") pod \"maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2\" (UID: \"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"
Oct 30 19:05:18.928790 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.928705    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-maas-api-tls\") pod \"maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2\" (UID: \"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"
Oct 30 19:05:18.928979 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:05:18.928860    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-serving-cert: secret "maas-api-e2e-ipp-a-1acf9b-serving-cert" not found
Oct 30 19:05:18.928979 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:05:18.928909    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-maas-api-tls podName:7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39 nodeName:}" failed. No retries permitted until 2026-10-30 19:05:19.42889362 +0000 UTC m=+1840.140642308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-maas-api-tls") pod "maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2" (UID: "7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39") : secret "maas-api-e2e-ipp-a-1acf9b-serving-cert" not found
Oct 30 19:05:18.929388 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.929365    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2\" (UID: \"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"
Oct 30 19:05:18.931286 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:18.931255    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-metrics-certs\") pod \"maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2\" (UID: \"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"
Oct 30 19:05:19.027767 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:19.021899    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4" podStartSLOduration=2.02187707 podStartE2EDuration="2.02187707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:19.018508726 +0000 UTC m=+1839.730257424" watchObservedRunningTime="2026-10-30 19:05:19.02187707 +0000 UTC m=+1839.733625768"
Oct 30 19:05:19.027767 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:19.023979    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr858\" (UniqueName: \"kubernetes.io/projected/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-kube-api-access-sr858\") pod \"maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2\" (UID: \"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"
Oct 30 19:05:19.031274 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:19.031240    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:19.422224 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:19.422182    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:19.437398 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:19.437367    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe" path="/var/lib/kubelet/pods/408a2e0d-e13c-4f8a-8f3b-9d16db19d9fe/volumes"
Oct 30 19:05:19.437804 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:19.437773    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-maas-api-tls\") pod \"maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2\" (UID: \"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"
Oct 30 19:05:19.439394 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:19.439368    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e4b29a58-09b8-4ef8-be55-49dcac8dc6a1" path="/var/lib/kubelet/pods/e4b29a58-09b8-4ef8-be55-49dcac8dc6a1/volumes"
Oct 30 19:05:19.441280 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:19.441257    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-maas-api-tls\") pod \"maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2\" (UID: \"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"
Oct 30 19:05:19.646416 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:19.646384    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"
Oct 30 19:05:19.698021 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:19.696454    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:20.072709 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:20.072678    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"]
Oct 30 19:05:20.543096 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:20.542998    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2" event={"ID":"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39","Type":"ContainerStarted","Data":"25e910a065ea4c70ccf2ddbdfed2b02f75798b720981d58b4fb5ef274e88fe33"}
Oct 30 19:05:21.551870 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:21.551829    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2" event={"ID":"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39","Type":"ContainerStarted","Data":"ded21665c7125e94dabe5f7ded56cea8e08bd2322bb1b0b74f66fcc0295eb742"}
Oct 30 19:05:21.552348 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:21.552060    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"
Oct 30 19:05:21.684167 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:21.684112    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2" podStartSLOduration=2.891379903 podStartE2EDuration="3.684098636s" totalImagesPullingTime="792.718733ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:21.682011861 +0000 UTC m=+1842.393760559" watchObservedRunningTime="2026-10-30 19:05:21.684098636 +0000 UTC m=+1842.395847333"
Oct 30 19:05:24.276233 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:24.276195    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:27.564336 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:27.564305    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"
Oct 30 19:05:29.155988 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:29.153600    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:29.538191 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:29.538118    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4"
Oct 30 19:05:29.547834 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:29.547812    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs"
Oct 30 19:05:31.625330 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:31.625289    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:31.919485 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:31.919167    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:34.015450 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:34.015414    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:38.204433 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:05:38.204385    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:05:39.074868 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:39.074827    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:45.542833 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:45.542788    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:50.428512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:50.428192    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:55.179736 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:55.179693    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:56.970066 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:05:56.970017    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:00.152140 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:00.152101    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:03.682361 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:03.682267    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:04.148112 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:04.148069    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:06.444571 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:06.444535    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"]
Oct 30 19:06:06.449019 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:06.448999    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"
Oct 30 19:06:06.501122 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:06.501095    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-e67448-serving-cert\""
Oct 30 19:06:06.553750 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:06.553713    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"]
Oct 30 19:06:06.603215 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:06.603184    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2xdn\" (UniqueName: \"kubernetes.io/projected/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-kube-api-access-q2xdn\") pod \"maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk\" (UID: \"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"
Oct 30 19:06:06.603369 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:06.603240    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-openshift-service-ca\") pod \"maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk\" (UID: \"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"
Oct 30 19:06:06.603436 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:06.603415    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-metrics-certs\") pod \"maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk\" (UID: \"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"
Oct 30 19:06:06.603565 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:06.603545    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-maas-api-tls\") pod \"maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk\" (UID: \"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"
Oct 30 19:06:06.687809 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:06.687761    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:06.705020 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:06.704919    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-metrics-certs\") pod \"maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk\" (UID: \"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"
Oct 30 19:06:06.705142 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:06.705071    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-maas-api-tls\") pod \"maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk\" (UID: \"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"
Oct 30 19:06:06.705182 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:06.705167    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2xdn\" (UniqueName: \"kubernetes.io/projected/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-kube-api-access-q2xdn\") pod \"maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk\" (UID: \"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"
Oct 30 19:06:06.705235 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:06.705218    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-openshift-service-ca\") pod \"maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk\" (UID: \"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"
Oct 30 19:06:06.705857 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:06.705824    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-openshift-service-ca\") pod \"maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk\" (UID: \"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"
Oct 30 19:06:06.707533 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:06.707511    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-metrics-certs\") pod \"maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk\" (UID: \"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"
Oct 30 19:06:06.707633 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:06.707554    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-maas-api-tls\") pod \"maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk\" (UID: \"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"
Oct 30 19:06:06.766397 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:06.766364    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2xdn\" (UniqueName: \"kubernetes.io/projected/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-kube-api-access-q2xdn\") pod \"maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk\" (UID: \"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"
Oct 30 19:06:06.884167 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:06.884069    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:07.063025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:07.062997    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"
Oct 30 19:06:07.444715 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:07.444691    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"]
Oct 30 19:06:07.445162 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:06:07.445142    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5ef49d3_3505_4ae5_aa39_ddeebffa35fd.slice/crio-85b3a0e92070df3cbe4955b572e6a21df07fb6627624478315ef0c250ae5eff4 WatchSource:0}: Error finding container 85b3a0e92070df3cbe4955b572e6a21df07fb6627624478315ef0c250ae5eff4: Status 404 returned error can't find the container with id 85b3a0e92070df3cbe4955b572e6a21df07fb6627624478315ef0c250ae5eff4
Oct 30 19:06:07.446473 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:07.446452    5820 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:06:07.919673 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:07.919635    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk" event={"ID":"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd","Type":"ContainerStarted","Data":"85b3a0e92070df3cbe4955b572e6a21df07fb6627624478315ef0c250ae5eff4"}
Oct 30 19:06:08.204095 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:06:08.204010    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:06:08.930263 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:08.930221    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk" event={"ID":"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd","Type":"ContainerStarted","Data":"fe8107be5499c32a293ca5bea7078e3c1c9862bb226bea0dc4d72e2ed4b54de2"}
Oct 30 19:06:08.930702 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:08.930438    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"
Oct 30 19:06:09.056399 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:09.056352    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk" podStartSLOduration=2.189883988 podStartE2EDuration="3.05633619s" totalImagesPullingTime="866.452202ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:09.053761609 +0000 UTC m=+1889.765510296" watchObservedRunningTime="2026-10-30 19:06:09.05633619 +0000 UTC m=+1889.768084886"
Oct 30 19:06:11.564295 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:11.564242    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:11.808219 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:11.808062    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:11.985268 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:11.985195    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:14.942240 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:14.942211    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"
Oct 30 19:06:16.546061 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:16.546018    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:16.649926 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:16.649882    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"]
Oct 30 19:06:16.650227 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:16.650201    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf" podUID="76921ad3-c8c7-4de8-b9ef-5198a8c94327" containerName="istio-proxy" containerID="cri-o://7ac5e6072471f1654d6ffa6bf7ed1abaa8f3818b44ae7078bbcb1361c390ee11" gracePeriod=30
Oct 30 19:06:21.443255 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:21.442825    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:21.918697 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:21.918670    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 19:06:22.017422 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.017343    5820 generic.go:356] "Generic (PLEG): container finished" podID="76921ad3-c8c7-4de8-b9ef-5198a8c94327" containerID="7ac5e6072471f1654d6ffa6bf7ed1abaa8f3818b44ae7078bbcb1361c390ee11" exitCode=0
Oct 30 19:06:22.017422 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.017390    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"
Oct 30 19:06:22.017590 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.017429    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf" event={"ID":"76921ad3-c8c7-4de8-b9ef-5198a8c94327","Type":"ContainerDied","Data":"7ac5e6072471f1654d6ffa6bf7ed1abaa8f3818b44ae7078bbcb1361c390ee11"}
Oct 30 19:06:22.017590 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.017483    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf" event={"ID":"76921ad3-c8c7-4de8-b9ef-5198a8c94327","Type":"ContainerDied","Data":"70f19cc2875d9b1b4763925f4ca247fccc39d36a42caf35c100239d5351abf9f"}
Oct 30 19:06:22.017590 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.017506    5820 scope.go:122] "RemoveContainer" containerID="7ac5e6072471f1654d6ffa6bf7ed1abaa8f3818b44ae7078bbcb1361c390ee11"
Oct 30 19:06:22.027939 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.027922    5820 scope.go:122] "RemoveContainer" containerID="7ac5e6072471f1654d6ffa6bf7ed1abaa8f3818b44ae7078bbcb1361c390ee11"
Oct 30 19:06:22.028288 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:06:22.028268    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ac5e6072471f1654d6ffa6bf7ed1abaa8f3818b44ae7078bbcb1361c390ee11\": container with ID starting with 7ac5e6072471f1654d6ffa6bf7ed1abaa8f3818b44ae7078bbcb1361c390ee11 not found: ID does not exist" containerID="7ac5e6072471f1654d6ffa6bf7ed1abaa8f3818b44ae7078bbcb1361c390ee11"
Oct 30 19:06:22.028345 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.028301    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ac5e6072471f1654d6ffa6bf7ed1abaa8f3818b44ae7078bbcb1361c390ee11"} err="failed to get container status \"7ac5e6072471f1654d6ffa6bf7ed1abaa8f3818b44ae7078bbcb1361c390ee11\": rpc error: code = NotFound desc = could not find container \"7ac5e6072471f1654d6ffa6bf7ed1abaa8f3818b44ae7078bbcb1361c390ee11\": container with ID starting with 7ac5e6072471f1654d6ffa6bf7ed1abaa8f3818b44ae7078bbcb1361c390ee11 not found: ID does not exist"
Oct 30 19:06:22.036505 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.036483    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-credential-socket\") pod \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") "
Oct 30 19:06:22.036609 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.036542    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-workload-socket\") pod \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") "
Oct 30 19:06:22.036609 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.036558    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-envoy\") pod \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") "
Oct 30 19:06:22.036609 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.036583    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76921ad3-c8c7-4de8-b9ef-5198a8c94327-kube-api-access-trnkw\" (UniqueName: \"kubernetes.io/projected/76921ad3-c8c7-4de8-b9ef-5198a8c94327-kube-api-access-trnkw\") pod \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") "
Oct 30 19:06:22.036782 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.036637    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-data\") pod \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") "
Oct 30 19:06:22.036782 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.036699    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-token\" (UniqueName: \"kubernetes.io/projected/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-token\") pod \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") "
Oct 30 19:06:22.036782 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.036747    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-workload-certs\") pod \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") "
Oct 30 19:06:22.036782 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.036753    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-credential-socket" pod "76921ad3-c8c7-4de8-b9ef-5198a8c94327" (UID: "76921ad3-c8c7-4de8-b9ef-5198a8c94327"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:22.037449 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.036849    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istiod-ca-cert\") pod \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") "
Oct 30 19:06:22.037449 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.036880    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-workload-socket" pod "76921ad3-c8c7-4de8-b9ef-5198a8c94327" (UID: "76921ad3-c8c7-4de8-b9ef-5198a8c94327"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:22.037449 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.036938    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-podinfo\") pod \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\" (UID: \"76921ad3-c8c7-4de8-b9ef-5198a8c94327\") "
Oct 30 19:06:22.037449 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.036970    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-workload-certs" pod "76921ad3-c8c7-4de8-b9ef-5198a8c94327" (UID: "76921ad3-c8c7-4de8-b9ef-5198a8c94327"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:22.037449 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.036982    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-data" pod "76921ad3-c8c7-4de8-b9ef-5198a8c94327" (UID: "76921ad3-c8c7-4de8-b9ef-5198a8c94327"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:22.037449 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.037229    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istiod-ca-cert" pod "76921ad3-c8c7-4de8-b9ef-5198a8c94327" (UID: "76921ad3-c8c7-4de8-b9ef-5198a8c94327"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:06:22.037697 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.037479    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:22.037697 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.037498    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:22.037697 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.037512    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:22.037697 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.037529    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:22.037697 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.037544    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:22.039045 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.039019    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-token" pod "76921ad3-c8c7-4de8-b9ef-5198a8c94327" (UID: "76921ad3-c8c7-4de8-b9ef-5198a8c94327"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:22.039193 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.039175    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-podinfo" pod "76921ad3-c8c7-4de8-b9ef-5198a8c94327" (UID: "76921ad3-c8c7-4de8-b9ef-5198a8c94327"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:06:22.039334 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.039318    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-envoy" pod "76921ad3-c8c7-4de8-b9ef-5198a8c94327" (UID: "76921ad3-c8c7-4de8-b9ef-5198a8c94327"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:22.039392 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.039339    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76921ad3-c8c7-4de8-b9ef-5198a8c94327-kube-api-access-trnkw" pod "76921ad3-c8c7-4de8-b9ef-5198a8c94327" (UID: "76921ad3-c8c7-4de8-b9ef-5198a8c94327"). InnerVolumeSpecName "kube-api-access-trnkw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:22.139426 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.139394    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:22.139426 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.139424    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:22.139622 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.139437    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trnkw\" (UniqueName: \"kubernetes.io/projected/76921ad3-c8c7-4de8-b9ef-5198a8c94327-kube-api-access-trnkw\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:22.139622 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.139450    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76921ad3-c8c7-4de8-b9ef-5198a8c94327-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:22.439742 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.439709    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"]
Oct 30 19:06:22.443999 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:22.443971    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-qxlbf"]
Oct 30 19:06:23.437655 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:23.437622    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="76921ad3-c8c7-4de8-b9ef-5198a8c94327" path="/var/lib/kubelet/pods/76921ad3-c8c7-4de8-b9ef-5198a8c94327/volumes"
Oct 30 19:06:23.631323 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:23.631284    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:24.109374 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:24.109317    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:25.454787 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:25.454755    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj"]
Oct 30 19:06:25.858700 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:25.858649    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:26.091448 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:26.091414    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:26.328268 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:26.328239    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"]
Oct 30 19:06:26.328938 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:26.328925    5820 cpu_manager.go:414] "containerMap: removing container" podUID="76921ad3-c8c7-4de8-b9ef-5198a8c94327" containerName="istio-proxy"
Oct 30 19:06:26.329001 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:26.328941    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76921ad3-c8c7-4de8-b9ef-5198a8c94327" containerName="istio-proxy"
Oct 30 19:06:26.329080 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:26.329070    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="76921ad3-c8c7-4de8-b9ef-5198a8c94327" containerName="istio-proxy"
Oct 30 19:06:26.334071 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:26.334048    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"
Oct 30 19:06:26.388237 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:26.388203    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-0ee379-serving-cert\""
Oct 30 19:06:26.392299 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:26.392274    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"]
Oct 30 19:06:26.413914 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:26.413890    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c143e1e4-afe7-48ea-b014-d586f51bcacf-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7\" (UID: \"c143e1e4-afe7-48ea-b014-d586f51bcacf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"
Oct 30 19:06:26.414054 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:26.413925    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c143e1e4-afe7-48ea-b014-d586f51bcacf-metrics-certs\") pod \"maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7\" (UID: \"c143e1e4-afe7-48ea-b014-d586f51bcacf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"
Oct 30 19:06:26.414054 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:26.413978    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x6bx\" (UniqueName: \"kubernetes.io/projected/c143e1e4-afe7-48ea-b014-d586f51bcacf-kube-api-access-9x6bx\") pod \"maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7\" (UID: \"c143e1e4-afe7-48ea-b014-d586f51bcacf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"
Oct 30 19:06:26.414179 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:26.414128    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c143e1e4-afe7-48ea-b014-d586f51bcacf-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7\" (UID: \"c143e1e4-afe7-48ea-b014-d586f51bcacf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"
Oct 30 19:06:26.515483 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:26.515450    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c143e1e4-afe7-48ea-b014-d586f51bcacf-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7\" (UID: \"c143e1e4-afe7-48ea-b014-d586f51bcacf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"
Oct 30 19:06:26.515977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:26.515488    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c143e1e4-afe7-48ea-b014-d586f51bcacf-metrics-certs\") pod \"maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7\" (UID: \"c143e1e4-afe7-48ea-b014-d586f51bcacf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"
Oct 30 19:06:26.515977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:26.515524    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x6bx\" (UniqueName: \"kubernetes.io/projected/c143e1e4-afe7-48ea-b014-d586f51bcacf-kube-api-access-9x6bx\") pod \"maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7\" (UID: \"c143e1e4-afe7-48ea-b014-d586f51bcacf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"
Oct 30 19:06:26.515977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:26.515577    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c143e1e4-afe7-48ea-b014-d586f51bcacf-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7\" (UID: \"c143e1e4-afe7-48ea-b014-d586f51bcacf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"
Oct 30 19:06:26.515977 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:06:26.515612    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-serving-cert: secret "maas-api-e2e-ipp-b-0ee379-serving-cert" not found
Oct 30 19:06:26.515977 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:06:26.515678    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c143e1e4-afe7-48ea-b014-d586f51bcacf-maas-api-tls podName:c143e1e4-afe7-48ea-b014-d586f51bcacf nodeName:}" failed. No retries permitted until 2026-10-30 19:06:27.015658364 +0000 UTC m=+1907.727407067 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c143e1e4-afe7-48ea-b014-d586f51bcacf-maas-api-tls") pod "maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7" (UID: "c143e1e4-afe7-48ea-b014-d586f51bcacf") : secret "maas-api-e2e-ipp-b-0ee379-serving-cert" not found
Oct 30 19:06:26.516462 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:26.516441    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c143e1e4-afe7-48ea-b014-d586f51bcacf-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7\" (UID: \"c143e1e4-afe7-48ea-b014-d586f51bcacf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"
Oct 30 19:06:26.518136 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:26.518114    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c143e1e4-afe7-48ea-b014-d586f51bcacf-metrics-certs\") pod \"maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7\" (UID: \"c143e1e4-afe7-48ea-b014-d586f51bcacf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"
Oct 30 19:06:26.566446 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:26.566409    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:26.622526 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:26.622247    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x6bx\" (UniqueName: \"kubernetes.io/projected/c143e1e4-afe7-48ea-b014-d586f51bcacf-kube-api-access-9x6bx\") pod \"maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7\" (UID: \"c143e1e4-afe7-48ea-b014-d586f51bcacf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"
Oct 30 19:06:26.885493 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:26.885412    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:27.037972 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:27.031933    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c143e1e4-afe7-48ea-b014-d586f51bcacf-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7\" (UID: \"c143e1e4-afe7-48ea-b014-d586f51bcacf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"
Oct 30 19:06:27.051971 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:27.048779    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c143e1e4-afe7-48ea-b014-d586f51bcacf-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7\" (UID: \"c143e1e4-afe7-48ea-b014-d586f51bcacf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"
Oct 30 19:06:27.067624 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:27.067589    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:27.249434 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:27.248901    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"
Oct 30 19:06:27.643222 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:27.643195    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"]
Oct 30 19:06:27.644600 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:06:27.644558    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc143e1e4_afe7_48ea_b014_d586f51bcacf.slice/crio-4f7aa7e1b5bee9c34f89e5e606b8705c5d9bca602b4757851d1c2e2bcc1ce436 WatchSource:0}: Error finding container 4f7aa7e1b5bee9c34f89e5e606b8705c5d9bca602b4757851d1c2e2bcc1ce436: Status 404 returned error can't find the container with id 4f7aa7e1b5bee9c34f89e5e606b8705c5d9bca602b4757851d1c2e2bcc1ce436
Oct 30 19:06:28.075801 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:28.075770    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7" event={"ID":"c143e1e4-afe7-48ea-b014-d586f51bcacf","Type":"ContainerStarted","Data":"4f7aa7e1b5bee9c34f89e5e606b8705c5d9bca602b4757851d1c2e2bcc1ce436"}
Oct 30 19:06:29.102976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:29.092443    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7" event={"ID":"c143e1e4-afe7-48ea-b014-d586f51bcacf","Type":"ContainerStarted","Data":"19327e77c5a049484e7659c9966dd7fb0f5b014a52766a543d01ebcac17ef037"}
Oct 30 19:06:29.102976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:29.093566    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"
Oct 30 19:06:29.218715 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:29.217936    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7" podStartSLOduration=2.405404701 podStartE2EDuration="3.217918002s" totalImagesPullingTime="812.513301ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:29.217326981 +0000 UTC m=+1909.929075682" watchObservedRunningTime="2026-10-30 19:06:29.217918002 +0000 UTC m=+1909.929666699"
Oct 30 19:06:30.458978 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:30.455018    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj" podUID="92af8465-a779-4904-b08c-aea834e86824" containerName="payload-processing" containerID="cri-o://7d4209daed8cc811636e022eb81b83f826f9110d20925323a49d00b0a1b1bbe7" gracePeriod=25
Oct 30 19:06:30.780644 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:30.780608    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj"
Oct 30 19:06:30.924307 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:30.924274    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/92af8465-a779-4904-b08c-aea834e86824-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92af8465-a779-4904-b08c-aea834e86824-plugins-config-volume\") pod \"92af8465-a779-4904-b08c-aea834e86824\" (UID: \"92af8465-a779-4904-b08c-aea834e86824\") "
Oct 30 19:06:30.924307 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:30.924314    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92af8465-a779-4904-b08c-aea834e86824-kube-api-access-v55b4\" (UniqueName: \"kubernetes.io/projected/92af8465-a779-4904-b08c-aea834e86824-kube-api-access-v55b4\") pod \"92af8465-a779-4904-b08c-aea834e86824\" (UID: \"92af8465-a779-4904-b08c-aea834e86824\") "
Oct 30 19:06:30.924677 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:30.924654    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92af8465-a779-4904-b08c-aea834e86824-plugins-config-volume" pod "92af8465-a779-4904-b08c-aea834e86824" (UID: "92af8465-a779-4904-b08c-aea834e86824"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:06:30.924751 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:30.924737    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92af8465-a779-4904-b08c-aea834e86824-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:30.926621 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:30.926600    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92af8465-a779-4904-b08c-aea834e86824-kube-api-access-v55b4" pod "92af8465-a779-4904-b08c-aea834e86824" (UID: "92af8465-a779-4904-b08c-aea834e86824"). InnerVolumeSpecName "kube-api-access-v55b4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:31.026116 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:31.026026    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v55b4\" (UniqueName: \"kubernetes.io/projected/92af8465-a779-4904-b08c-aea834e86824-kube-api-access-v55b4\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:31.106717 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:31.106680    5820 generic.go:356] "Generic (PLEG): container finished" podID="92af8465-a779-4904-b08c-aea834e86824" containerID="7d4209daed8cc811636e022eb81b83f826f9110d20925323a49d00b0a1b1bbe7" exitCode=0
Oct 30 19:06:31.106910 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:31.106728    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj"
Oct 30 19:06:31.106910 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:31.106757    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj" event={"ID":"92af8465-a779-4904-b08c-aea834e86824","Type":"ContainerDied","Data":"7d4209daed8cc811636e022eb81b83f826f9110d20925323a49d00b0a1b1bbe7"}
Oct 30 19:06:31.106910 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:31.106826    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj" event={"ID":"92af8465-a779-4904-b08c-aea834e86824","Type":"ContainerDied","Data":"db4e7ee21ac6a270c12b49cb17f65af9989f4aebf8a9afdd5259f3af05e38c87"}
Oct 30 19:06:31.106910 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:31.106848    5820 scope.go:122] "RemoveContainer" containerID="7d4209daed8cc811636e022eb81b83f826f9110d20925323a49d00b0a1b1bbe7"
Oct 30 19:06:31.118174 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:31.118155    5820 scope.go:122] "RemoveContainer" containerID="7d4209daed8cc811636e022eb81b83f826f9110d20925323a49d00b0a1b1bbe7"
Oct 30 19:06:31.118457 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:06:31.118438    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d4209daed8cc811636e022eb81b83f826f9110d20925323a49d00b0a1b1bbe7\": container with ID starting with 7d4209daed8cc811636e022eb81b83f826f9110d20925323a49d00b0a1b1bbe7 not found: ID does not exist" containerID="7d4209daed8cc811636e022eb81b83f826f9110d20925323a49d00b0a1b1bbe7"
Oct 30 19:06:31.118496 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:31.118468    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d4209daed8cc811636e022eb81b83f826f9110d20925323a49d00b0a1b1bbe7"} err="failed to get container status \"7d4209daed8cc811636e022eb81b83f826f9110d20925323a49d00b0a1b1bbe7\": rpc error: code = NotFound desc = could not find container \"7d4209daed8cc811636e022eb81b83f826f9110d20925323a49d00b0a1b1bbe7\": container with ID starting with 7d4209daed8cc811636e022eb81b83f826f9110d20925323a49d00b0a1b1bbe7 not found: ID does not exist"
Oct 30 19:06:31.273097 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:31.273063    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj"]
Oct 30 19:06:31.275813 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:31.275793    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fee9f0-6bc86cc4f6-m9kwj"]
Oct 30 19:06:31.461840 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:31.461802    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="92af8465-a779-4904-b08c-aea834e86824" path="/var/lib/kubelet/pods/92af8465-a779-4904-b08c-aea834e86824/volumes"
Oct 30 19:06:32.095566 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:32.094258    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:36.113457 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:36.113424    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"
Oct 30 19:06:37.159976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:37.151225    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:38.211978 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:06:38.211917    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:06:39.216478 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:39.216382    5820 generic.go:356] "Generic (PLEG): container finished" podID="cda46604-6d15-4ee9-823f-cb5cb70f4c51" containerID="488e8b395f1d7baa8912b4bda18e4ba773af44820105b23c6943603e130aa9df" exitCode=0
Oct 30 19:06:39.216478 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:39.216452    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98" event={"ID":"cda46604-6d15-4ee9-823f-cb5cb70f4c51","Type":"ContainerDied","Data":"488e8b395f1d7baa8912b4bda18e4ba773af44820105b23c6943603e130aa9df"}
Oct 30 19:06:39.216999 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:39.216908    5820 scope.go:122] "RemoveContainer" containerID="488e8b395f1d7baa8912b4bda18e4ba773af44820105b23c6943603e130aa9df"
Oct 30 19:06:39.218580 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:39.218548    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:06:39.218641 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:39.218618    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:06:39.218685 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:39.218651    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:06:40.225163 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:40.225128    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98" event={"ID":"cda46604-6d15-4ee9-823f-cb5cb70f4c51","Type":"ContainerStarted","Data":"b437d90c6fa5ae131fb8ae9af8bf2ca377a3d06e4799dac75ee77ba890e48e82"}
Oct 30 19:06:40.533370 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:40.533343    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:06:40.533556 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:40.533516    5820 patch_prober.go:29] interesting pod/e2e-api-b-e67448-openshift-default-568d47b899-v6w98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 19:06:40.533602 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:40.533553    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98" podUID="cda46604-6d15-4ee9-823f-cb5cb70f4c51" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 19:06:41.374162 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:41.374128    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf"]
Oct 30 19:06:41.417396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:41.417355    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:41.540976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:41.534303    5820 patch_prober.go:29] interesting pod/e2e-api-b-e67448-openshift-default-568d47b899-v6w98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 19:06:41.540976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:41.534360    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98" podUID="cda46604-6d15-4ee9-823f-cb5cb70f4c51" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 19:06:41.791306 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:41.791221    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:42.050046 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:42.024573    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:42.527036 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:42.523929    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:42.543030 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:42.536274    5820 patch_prober.go:29] interesting pod/e2e-api-b-e67448-openshift-default-568d47b899-v6w98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 19:06:42.543030 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:42.536373    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98" podUID="cda46604-6d15-4ee9-823f-cb5cb70f4c51" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 19:06:43.533891 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:43.533855    5820 patch_prober.go:29] interesting pod/e2e-api-b-e67448-openshift-default-568d47b899-v6w98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 19:06:43.534397 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:43.533929    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98" podUID="cda46604-6d15-4ee9-823f-cb5cb70f4c51" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 19:06:44.543976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:44.538329    5820 patch_prober.go:29] interesting pod/e2e-api-b-e67448-openshift-default-568d47b899-v6w98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 19:06:44.543976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:44.538392    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98" podUID="cda46604-6d15-4ee9-823f-cb5cb70f4c51" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 19:06:45.615060 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:45.615027    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:06:45.615900 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:45.615874    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:06:45.616805 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:45.616776    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:06:46.375717 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:46.375673    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf" podUID="5fca5c67-64a5-44f6-9fb1-59d47874e39f" containerName="payload-processing" containerID="cri-o://5de4ac45090f024ec81cf7c82946009ef96e561e6ac1cbeb69fe3302c47c0680" gracePeriod=25
Oct 30 19:06:46.657914 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:46.657889    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf"
Oct 30 19:06:46.789814 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:46.789731    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5fca5c67-64a5-44f6-9fb1-59d47874e39f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fca5c67-64a5-44f6-9fb1-59d47874e39f-plugins-config-volume\") pod \"5fca5c67-64a5-44f6-9fb1-59d47874e39f\" (UID: \"5fca5c67-64a5-44f6-9fb1-59d47874e39f\") "
Oct 30 19:06:46.790010 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:46.789878    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5fca5c67-64a5-44f6-9fb1-59d47874e39f-kube-api-access-229mr\" (UniqueName: \"kubernetes.io/projected/5fca5c67-64a5-44f6-9fb1-59d47874e39f-kube-api-access-229mr\") pod \"5fca5c67-64a5-44f6-9fb1-59d47874e39f\" (UID: \"5fca5c67-64a5-44f6-9fb1-59d47874e39f\") "
Oct 30 19:06:46.790161 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:46.790132    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fca5c67-64a5-44f6-9fb1-59d47874e39f-plugins-config-volume" pod "5fca5c67-64a5-44f6-9fb1-59d47874e39f" (UID: "5fca5c67-64a5-44f6-9fb1-59d47874e39f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:06:46.790430 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:46.790412    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fca5c67-64a5-44f6-9fb1-59d47874e39f-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:46.792227 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:46.792197    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fca5c67-64a5-44f6-9fb1-59d47874e39f-kube-api-access-229mr" pod "5fca5c67-64a5-44f6-9fb1-59d47874e39f" (UID: "5fca5c67-64a5-44f6-9fb1-59d47874e39f"). InnerVolumeSpecName "kube-api-access-229mr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:46.892052 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:46.891976    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-229mr\" (UniqueName: \"kubernetes.io/projected/5fca5c67-64a5-44f6-9fb1-59d47874e39f-kube-api-access-229mr\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:47.300868 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:47.300830    5820 generic.go:356] "Generic (PLEG): container finished" podID="5fca5c67-64a5-44f6-9fb1-59d47874e39f" containerID="5de4ac45090f024ec81cf7c82946009ef96e561e6ac1cbeb69fe3302c47c0680" exitCode=0
Oct 30 19:06:47.301059 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:47.300880    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf"
Oct 30 19:06:47.301059 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:47.300917    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf" event={"ID":"5fca5c67-64a5-44f6-9fb1-59d47874e39f","Type":"ContainerDied","Data":"5de4ac45090f024ec81cf7c82946009ef96e561e6ac1cbeb69fe3302c47c0680"}
Oct 30 19:06:47.301059 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:47.300991    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf" event={"ID":"5fca5c67-64a5-44f6-9fb1-59d47874e39f","Type":"ContainerDied","Data":"0b19fb78f3c8f80a586f8a17800c03d78bf1a7a1f908463748a0b5b3162b8b46"}
Oct 30 19:06:47.301059 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:47.301023    5820 scope.go:122] "RemoveContainer" containerID="5de4ac45090f024ec81cf7c82946009ef96e561e6ac1cbeb69fe3302c47c0680"
Oct 30 19:06:47.302647 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:47.302624    5820 generic.go:356] "Generic (PLEG): container finished" podID="3fd20f13-a79c-4b16-b85c-56827c10e159" containerID="454a8da06583e19d770cffddac86965d58f09eddd5ccf031fcf3cb3f7ee0c243" exitCode=0
Oct 30 19:06:47.302745 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:47.302698    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp" event={"ID":"3fd20f13-a79c-4b16-b85c-56827c10e159","Type":"ContainerDied","Data":"454a8da06583e19d770cffddac86965d58f09eddd5ccf031fcf3cb3f7ee0c243"}
Oct 30 19:06:47.303203 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:47.303185    5820 scope.go:122] "RemoveContainer" containerID="454a8da06583e19d770cffddac86965d58f09eddd5ccf031fcf3cb3f7ee0c243"
Oct 30 19:06:47.304646 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:47.304616    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:06:47.304790 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:47.304692    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:06:47.304790 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:47.304747    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:06:47.311747 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:47.311724    5820 scope.go:122] "RemoveContainer" containerID="5de4ac45090f024ec81cf7c82946009ef96e561e6ac1cbeb69fe3302c47c0680"
Oct 30 19:06:47.312034 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:06:47.312016    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5de4ac45090f024ec81cf7c82946009ef96e561e6ac1cbeb69fe3302c47c0680\": container with ID starting with 5de4ac45090f024ec81cf7c82946009ef96e561e6ac1cbeb69fe3302c47c0680 not found: ID does not exist" containerID="5de4ac45090f024ec81cf7c82946009ef96e561e6ac1cbeb69fe3302c47c0680"
Oct 30 19:06:47.312105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:47.312044    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5de4ac45090f024ec81cf7c82946009ef96e561e6ac1cbeb69fe3302c47c0680"} err="failed to get container status \"5de4ac45090f024ec81cf7c82946009ef96e561e6ac1cbeb69fe3302c47c0680\": rpc error: code = NotFound desc = could not find container \"5de4ac45090f024ec81cf7c82946009ef96e561e6ac1cbeb69fe3302c47c0680\": container with ID starting with 5de4ac45090f024ec81cf7c82946009ef96e561e6ac1cbeb69fe3302c47c0680 not found: ID does not exist"
Oct 30 19:06:47.592358 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:47.592056    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf"]
Oct 30 19:06:47.598625 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:47.598592    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-1acf9b-859fc76f64-t4bhf"]
Oct 30 19:06:47.719976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:47.717291    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:48.310848 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:48.310811    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp" event={"ID":"3fd20f13-a79c-4b16-b85c-56827c10e159","Type":"ContainerStarted","Data":"e39b7b83b408119cde6355c962503762bc8177278adcc106402767d9425f1ec8"}
Oct 30 19:06:48.876268 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:48.876232    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:06:48.877909 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:48.877883    5820 patch_prober.go:29] interesting pod/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 19:06:48.878052 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:48.877932    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp" podUID="3fd20f13-a79c-4b16-b85c-56827c10e159" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 19:06:49.458976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:49.455022    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5fca5c67-64a5-44f6-9fb1-59d47874e39f" path="/var/lib/kubelet/pods/5fca5c67-64a5-44f6-9fb1-59d47874e39f/volumes"
Oct 30 19:06:49.875922 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:49.875882    5820 patch_prober.go:29] interesting pod/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 19:06:49.876123 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:49.875994    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp" podUID="3fd20f13-a79c-4b16-b85c-56827c10e159" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 19:06:50.323280 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:50.323232    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:50.565974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:50.562814    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:50.876038 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:50.875998    5820 patch_prober.go:29] interesting pod/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 19:06:50.876228 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:50.876098    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp" podUID="3fd20f13-a79c-4b16-b85c-56827c10e159" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 19:06:51.767889 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:51.767857    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:06:51.876205 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:51.876172    5820 patch_prober.go:29] interesting pod/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 19:06:51.876374 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:51.876237    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp" podUID="3fd20f13-a79c-4b16-b85c-56827c10e159" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 19:06:51.888209 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:51.888177    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"]
Oct 30 19:06:51.888436 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:51.888412    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj" podUID="f0408420-508d-475e-a6cb-85e9dfc06d4b" containerName="maas-api" containerID="cri-o://b182c40ea8a5e7ba9a8be064d80a5d88f0c677a0722d861224902b9c1ea73853" gracePeriod=30
Oct 30 19:06:52.063009 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.062894    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:52.367974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.359464    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg"]
Oct 30 19:06:52.367974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.359699    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg" podUID="db22814d-f854-4959-8f90-3c0e667ab59b" containerName="payload-processing" containerID="cri-o://bfadc79342a99bebf8fae041524878f319f1390f3eb591fcf0321dda0c316738" gracePeriod=30
Oct 30 19:06:52.371974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.371933    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"
Oct 30 19:06:52.373637 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.373614    5820 generic.go:356] "Generic (PLEG): container finished" podID="f0408420-508d-475e-a6cb-85e9dfc06d4b" containerID="b182c40ea8a5e7ba9a8be064d80a5d88f0c677a0722d861224902b9c1ea73853" exitCode=0
Oct 30 19:06:52.373730 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.373686    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj" event={"ID":"f0408420-508d-475e-a6cb-85e9dfc06d4b","Type":"ContainerDied","Data":"b182c40ea8a5e7ba9a8be064d80a5d88f0c677a0722d861224902b9c1ea73853"}
Oct 30 19:06:52.373780 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.373746    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj" event={"ID":"f0408420-508d-475e-a6cb-85e9dfc06d4b","Type":"ContainerDied","Data":"8b9ab48cad96d0f1a6788c2f7372d4431b95cb78e035909bcb9b58ec0b1e5114"}
Oct 30 19:06:52.373780 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.373769    5820 scope.go:122] "RemoveContainer" containerID="b182c40ea8a5e7ba9a8be064d80a5d88f0c677a0722d861224902b9c1ea73853"
Oct 30 19:06:52.397590 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.397567    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x"]
Oct 30 19:06:52.397772 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.397752    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x" podUID="fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d" containerName="payload-pre-processing" containerID="cri-o://65addbcc80b735c26b7609316dff1e5fbd59611012d830b58ae0e5e519e09bce" gracePeriod=30
Oct 30 19:06:52.462360 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.449523    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0408420-508d-475e-a6cb-85e9dfc06d4b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0408420-508d-475e-a6cb-85e9dfc06d4b-openshift-service-ca\") pod \"f0408420-508d-475e-a6cb-85e9dfc06d4b\" (UID: \"f0408420-508d-475e-a6cb-85e9dfc06d4b\") "
Oct 30 19:06:52.462360 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.449579    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0408420-508d-475e-a6cb-85e9dfc06d4b-kube-api-access-x6gh7\" (UniqueName: \"kubernetes.io/projected/f0408420-508d-475e-a6cb-85e9dfc06d4b-kube-api-access-x6gh7\") pod \"f0408420-508d-475e-a6cb-85e9dfc06d4b\" (UID: \"f0408420-508d-475e-a6cb-85e9dfc06d4b\") "
Oct 30 19:06:52.462360 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.449738    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f0408420-508d-475e-a6cb-85e9dfc06d4b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0408420-508d-475e-a6cb-85e9dfc06d4b-metrics-certs\") pod \"f0408420-508d-475e-a6cb-85e9dfc06d4b\" (UID: \"f0408420-508d-475e-a6cb-85e9dfc06d4b\") "
Oct 30 19:06:52.462360 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.449770    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f0408420-508d-475e-a6cb-85e9dfc06d4b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0408420-508d-475e-a6cb-85e9dfc06d4b-maas-api-tls\") pod \"f0408420-508d-475e-a6cb-85e9dfc06d4b\" (UID: \"f0408420-508d-475e-a6cb-85e9dfc06d4b\") "
Oct 30 19:06:52.462360 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.456184    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0408420-508d-475e-a6cb-85e9dfc06d4b-openshift-service-ca" pod "f0408420-508d-475e-a6cb-85e9dfc06d4b" (UID: "f0408420-508d-475e-a6cb-85e9dfc06d4b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:06:52.473112 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.471535    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0408420-508d-475e-a6cb-85e9dfc06d4b-kube-api-access-x6gh7" pod "f0408420-508d-475e-a6cb-85e9dfc06d4b" (UID: "f0408420-508d-475e-a6cb-85e9dfc06d4b"). InnerVolumeSpecName "kube-api-access-x6gh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:52.479329 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.477643    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0408420-508d-475e-a6cb-85e9dfc06d4b-maas-api-tls" pod "f0408420-508d-475e-a6cb-85e9dfc06d4b" (UID: "f0408420-508d-475e-a6cb-85e9dfc06d4b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:06:52.479857 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.479804    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0408420-508d-475e-a6cb-85e9dfc06d4b-metrics-certs" pod "f0408420-508d-475e-a6cb-85e9dfc06d4b" (UID: "f0408420-508d-475e-a6cb-85e9dfc06d4b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:06:52.497928 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.497907    5820 scope.go:122] "RemoveContainer" containerID="b182c40ea8a5e7ba9a8be064d80a5d88f0c677a0722d861224902b9c1ea73853"
Oct 30 19:06:52.498354 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:06:52.498324    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b182c40ea8a5e7ba9a8be064d80a5d88f0c677a0722d861224902b9c1ea73853\": container with ID starting with b182c40ea8a5e7ba9a8be064d80a5d88f0c677a0722d861224902b9c1ea73853 not found: ID does not exist" containerID="b182c40ea8a5e7ba9a8be064d80a5d88f0c677a0722d861224902b9c1ea73853"
Oct 30 19:06:52.498473 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.498368    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b182c40ea8a5e7ba9a8be064d80a5d88f0c677a0722d861224902b9c1ea73853"} err="failed to get container status \"b182c40ea8a5e7ba9a8be064d80a5d88f0c677a0722d861224902b9c1ea73853\": rpc error: code = NotFound desc = could not find container \"b182c40ea8a5e7ba9a8be064d80a5d88f0c677a0722d861224902b9c1ea73853\": container with ID starting with b182c40ea8a5e7ba9a8be064d80a5d88f0c677a0722d861224902b9c1ea73853 not found: ID does not exist"
Oct 30 19:06:52.530590 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.530272    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x" podUID="fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.115:9004: connect: connection refused"
Oct 30 19:06:52.530734 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.530700    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg" podUID="db22814d-f854-4959-8f90-3c0e667ab59b" containerName="payload-processing" probeResult="failure" output="dial tcp 10.134.0.116:9004: connect: connection refused"
Oct 30 19:06:52.566290 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.551904    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0408420-508d-475e-a6cb-85e9dfc06d4b-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:52.566437 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.566301    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6gh7\" (UniqueName: \"kubernetes.io/projected/f0408420-508d-475e-a6cb-85e9dfc06d4b-kube-api-access-x6gh7\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:52.566437 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.566327    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0408420-508d-475e-a6cb-85e9dfc06d4b-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:52.566437 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.566337    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0408420-508d-475e-a6cb-85e9dfc06d4b-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:52.566807 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.566782    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:52.691596 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.691501    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg"
Oct 30 19:06:52.777426 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.777398    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db22814d-f854-4959-8f90-3c0e667ab59b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db22814d-f854-4959-8f90-3c0e667ab59b-plugins-config-volume\") pod \"db22814d-f854-4959-8f90-3c0e667ab59b\" (UID: \"db22814d-f854-4959-8f90-3c0e667ab59b\") "
Oct 30 19:06:52.777881 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.777450    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db22814d-f854-4959-8f90-3c0e667ab59b-kube-api-access-kdbdl\" (UniqueName: \"kubernetes.io/projected/db22814d-f854-4959-8f90-3c0e667ab59b-kube-api-access-kdbdl\") pod \"db22814d-f854-4959-8f90-3c0e667ab59b\" (UID: \"db22814d-f854-4959-8f90-3c0e667ab59b\") "
Oct 30 19:06:52.781552 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.781519    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db22814d-f854-4959-8f90-3c0e667ab59b-kube-api-access-kdbdl" pod "db22814d-f854-4959-8f90-3c0e667ab59b" (UID: "db22814d-f854-4959-8f90-3c0e667ab59b"). InnerVolumeSpecName "kube-api-access-kdbdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:52.781876 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.781852    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db22814d-f854-4959-8f90-3c0e667ab59b-plugins-config-volume" pod "db22814d-f854-4959-8f90-3c0e667ab59b" (UID: "db22814d-f854-4959-8f90-3c0e667ab59b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:06:52.784083 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.783939    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:52.786604 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.786581    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x"
Oct 30 19:06:52.876772 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.876678    5820 patch_prober.go:29] interesting pod/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 19:06:52.876772 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.876738    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp" podUID="3fd20f13-a79c-4b16-b85c-56827c10e159" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 19:06:52.878365 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.878339    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d-kube-api-access-5gllg\" (UniqueName: \"kubernetes.io/projected/fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d-kube-api-access-5gllg\") pod \"fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d\" (UID: \"fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d\") "
Oct 30 19:06:52.878502 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.878472    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d-plugins-config-volume\") pod \"fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d\" (UID: \"fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d\") "
Oct 30 19:06:52.878893 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.878860    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d-plugins-config-volume" pod "fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d" (UID: "fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:06:52.879072 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.879058    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db22814d-f854-4959-8f90-3c0e667ab59b-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:52.879132 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.879079    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdbdl\" (UniqueName: \"kubernetes.io/projected/db22814d-f854-4959-8f90-3c0e667ab59b-kube-api-access-kdbdl\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:52.879132 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.879093    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:52.880562 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.880533    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d-kube-api-access-5gllg" pod "fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d" (UID: "fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d"). InnerVolumeSpecName "kube-api-access-5gllg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:52.983777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:52.983721    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gllg\" (UniqueName: \"kubernetes.io/projected/fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d-kube-api-access-5gllg\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:53.382919 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.382872    5820 generic.go:356] "Generic (PLEG): container finished" podID="fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d" containerID="65addbcc80b735c26b7609316dff1e5fbd59611012d830b58ae0e5e519e09bce" exitCode=0
Oct 30 19:06:53.382919 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.382924    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x"
Oct 30 19:06:53.383244 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.382989    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x" event={"ID":"fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d","Type":"ContainerDied","Data":"65addbcc80b735c26b7609316dff1e5fbd59611012d830b58ae0e5e519e09bce"}
Oct 30 19:06:53.383244 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.383049    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x" event={"ID":"fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d","Type":"ContainerDied","Data":"e66c4e1be7025daab4e104cb1bc28a1b438671cc8b9062b4334627129f1df01a"}
Oct 30 19:06:53.383244 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.383073    5820 scope.go:122] "RemoveContainer" containerID="65addbcc80b735c26b7609316dff1e5fbd59611012d830b58ae0e5e519e09bce"
Oct 30 19:06:53.384916 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.384892    5820 generic.go:356] "Generic (PLEG): container finished" podID="db22814d-f854-4959-8f90-3c0e667ab59b" containerID="bfadc79342a99bebf8fae041524878f319f1390f3eb591fcf0321dda0c316738" exitCode=0
Oct 30 19:06:53.385085 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.384922    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg" event={"ID":"db22814d-f854-4959-8f90-3c0e667ab59b","Type":"ContainerDied","Data":"bfadc79342a99bebf8fae041524878f319f1390f3eb591fcf0321dda0c316738"}
Oct 30 19:06:53.385085 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.385004    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg" event={"ID":"db22814d-f854-4959-8f90-3c0e667ab59b","Type":"ContainerDied","Data":"75e0c4a34eb37d8ce88b8eb2d174667a00960f5c274c4502098211918cf83250"}
Oct 30 19:06:53.385085 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.385081    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg"
Oct 30 19:06:53.386457 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.386436    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"
Oct 30 19:06:53.396496 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.396479    5820 scope.go:122] "RemoveContainer" containerID="65addbcc80b735c26b7609316dff1e5fbd59611012d830b58ae0e5e519e09bce"
Oct 30 19:06:53.396754 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:06:53.396728    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65addbcc80b735c26b7609316dff1e5fbd59611012d830b58ae0e5e519e09bce\": container with ID starting with 65addbcc80b735c26b7609316dff1e5fbd59611012d830b58ae0e5e519e09bce not found: ID does not exist" containerID="65addbcc80b735c26b7609316dff1e5fbd59611012d830b58ae0e5e519e09bce"
Oct 30 19:06:53.396850 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.396770    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65addbcc80b735c26b7609316dff1e5fbd59611012d830b58ae0e5e519e09bce"} err="failed to get container status \"65addbcc80b735c26b7609316dff1e5fbd59611012d830b58ae0e5e519e09bce\": rpc error: code = NotFound desc = could not find container \"65addbcc80b735c26b7609316dff1e5fbd59611012d830b58ae0e5e519e09bce\": container with ID starting with 65addbcc80b735c26b7609316dff1e5fbd59611012d830b58ae0e5e519e09bce not found: ID does not exist"
Oct 30 19:06:53.396850 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.396791    5820 scope.go:122] "RemoveContainer" containerID="bfadc79342a99bebf8fae041524878f319f1390f3eb591fcf0321dda0c316738"
Oct 30 19:06:53.406606 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.406589    5820 scope.go:122] "RemoveContainer" containerID="bfadc79342a99bebf8fae041524878f319f1390f3eb591fcf0321dda0c316738"
Oct 30 19:06:53.406897 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:06:53.406876    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfadc79342a99bebf8fae041524878f319f1390f3eb591fcf0321dda0c316738\": container with ID starting with bfadc79342a99bebf8fae041524878f319f1390f3eb591fcf0321dda0c316738 not found: ID does not exist" containerID="bfadc79342a99bebf8fae041524878f319f1390f3eb591fcf0321dda0c316738"
Oct 30 19:06:53.406988 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.406908    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfadc79342a99bebf8fae041524878f319f1390f3eb591fcf0321dda0c316738"} err="failed to get container status \"bfadc79342a99bebf8fae041524878f319f1390f3eb591fcf0321dda0c316738\": rpc error: code = NotFound desc = could not find container \"bfadc79342a99bebf8fae041524878f319f1390f3eb591fcf0321dda0c316738\": container with ID starting with bfadc79342a99bebf8fae041524878f319f1390f3eb591fcf0321dda0c316738 not found: ID does not exist"
Oct 30 19:06:53.548392 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.548348    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"]
Oct 30 19:06:53.576236 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.576196    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:53.597612 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.597580    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-82305a-66bdfc9d75-ndhxj"]
Oct 30 19:06:53.660422 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.660342    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg"]
Oct 30 19:06:53.665621 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.665598    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-82305a-c95645884-hjxdg"]
Oct 30 19:06:53.774747 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.774711    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x"]
Oct 30 19:06:53.780359 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.780332    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-82305a-76ffb76bff-hrp5x"]
Oct 30 19:06:53.824250 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.824210    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:53.905842 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.905811    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:06:53.907639 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.907619    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:06:53.993905 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:53.993829    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:54.675798 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:54.675758    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:55.437714 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:55.437685    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="db22814d-f854-4959-8f90-3c0e667ab59b" path="/var/lib/kubelet/pods/db22814d-f854-4959-8f90-3c0e667ab59b/volumes"
Oct 30 19:06:55.438145 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:55.438130    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0408420-508d-475e-a6cb-85e9dfc06d4b" path="/var/lib/kubelet/pods/f0408420-508d-475e-a6cb-85e9dfc06d4b/volumes"
Oct 30 19:06:55.438502 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:55.438489    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d" path="/var/lib/kubelet/pods/fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d/volumes"
Oct 30 19:06:57.287770 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:57.287735    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:59.252255 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:06:59.252203    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:01.664942 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:01.664908    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:01.855907 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:01.855870    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:02.044089 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:02.044007    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:04.370759 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:04.370703    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:08.203433 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:07:08.203396    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:07:09.523928 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:09.523882    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:16.338435 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:16.338377    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:21.165890 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:21.165852    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:24.719468 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:24.719436    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:25.480662 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:25.480618    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"]
Oct 30 19:07:25.481614 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:25.481568    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4" podUID="e20e1010-00a5-490a-8aaa-93afaebe7932" containerName="maas-api" containerID="cri-o://119c611b32b963acf4f8f965a2dfb4526d0745ef038d649615175a78978c39a0" gracePeriod=30
Oct 30 19:07:25.607414 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:25.607370    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:25.804127 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:25.804105    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"
Oct 30 19:07:25.885646 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:25.878922    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e20e1010-00a5-490a-8aaa-93afaebe7932-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e20e1010-00a5-490a-8aaa-93afaebe7932-openshift-service-ca\") pod \"e20e1010-00a5-490a-8aaa-93afaebe7932\" (UID: \"e20e1010-00a5-490a-8aaa-93afaebe7932\") "
Oct 30 19:07:25.885646 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:25.879008    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e20e1010-00a5-490a-8aaa-93afaebe7932-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e20e1010-00a5-490a-8aaa-93afaebe7932-maas-api-tls\") pod \"e20e1010-00a5-490a-8aaa-93afaebe7932\" (UID: \"e20e1010-00a5-490a-8aaa-93afaebe7932\") "
Oct 30 19:07:25.885646 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:25.879223    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e20e1010-00a5-490a-8aaa-93afaebe7932-kube-api-access-cdpw6\" (UniqueName: \"kubernetes.io/projected/e20e1010-00a5-490a-8aaa-93afaebe7932-kube-api-access-cdpw6\") pod \"e20e1010-00a5-490a-8aaa-93afaebe7932\" (UID: \"e20e1010-00a5-490a-8aaa-93afaebe7932\") "
Oct 30 19:07:25.885646 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:25.879416    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e20e1010-00a5-490a-8aaa-93afaebe7932-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e20e1010-00a5-490a-8aaa-93afaebe7932-metrics-certs\") pod \"e20e1010-00a5-490a-8aaa-93afaebe7932\" (UID: \"e20e1010-00a5-490a-8aaa-93afaebe7932\") "
Oct 30 19:07:25.892119 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:25.892037    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e20e1010-00a5-490a-8aaa-93afaebe7932-openshift-service-ca" pod "e20e1010-00a5-490a-8aaa-93afaebe7932" (UID: "e20e1010-00a5-490a-8aaa-93afaebe7932"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:07:25.914337 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:25.911160    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e20e1010-00a5-490a-8aaa-93afaebe7932-maas-api-tls" pod "e20e1010-00a5-490a-8aaa-93afaebe7932" (UID: "e20e1010-00a5-490a-8aaa-93afaebe7932"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:07:25.916627 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:25.916567    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e20e1010-00a5-490a-8aaa-93afaebe7932-metrics-certs" pod "e20e1010-00a5-490a-8aaa-93afaebe7932" (UID: "e20e1010-00a5-490a-8aaa-93afaebe7932"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:07:25.916773 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:25.916679    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e20e1010-00a5-490a-8aaa-93afaebe7932-kube-api-access-cdpw6" pod "e20e1010-00a5-490a-8aaa-93afaebe7932" (UID: "e20e1010-00a5-490a-8aaa-93afaebe7932"). InnerVolumeSpecName "kube-api-access-cdpw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:25.981240 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:25.981205    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e20e1010-00a5-490a-8aaa-93afaebe7932-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:25.981240 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:25.981233    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e20e1010-00a5-490a-8aaa-93afaebe7932-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:25.981240 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:25.981245    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdpw6\" (UniqueName: \"kubernetes.io/projected/e20e1010-00a5-490a-8aaa-93afaebe7932-kube-api-access-cdpw6\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:25.981479 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:25.981255    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e20e1010-00a5-490a-8aaa-93afaebe7932-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:26.601156 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:26.601120    5820 generic.go:356] "Generic (PLEG): container finished" podID="e20e1010-00a5-490a-8aaa-93afaebe7932" containerID="119c611b32b963acf4f8f965a2dfb4526d0745ef038d649615175a78978c39a0" exitCode=0
Oct 30 19:07:26.601309 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:26.601224    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"
Oct 30 19:07:26.601309 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:26.601245    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4" event={"ID":"e20e1010-00a5-490a-8aaa-93afaebe7932","Type":"ContainerDied","Data":"119c611b32b963acf4f8f965a2dfb4526d0745ef038d649615175a78978c39a0"}
Oct 30 19:07:26.601397 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:26.601305    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4" event={"ID":"e20e1010-00a5-490a-8aaa-93afaebe7932","Type":"ContainerDied","Data":"fa45cf6191a4c51d372974271f0dc2296ce98e5fe41b61f1b29b8ce68a1ecb48"}
Oct 30 19:07:26.601397 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:26.601333    5820 scope.go:122] "RemoveContainer" containerID="119c611b32b963acf4f8f965a2dfb4526d0745ef038d649615175a78978c39a0"
Oct 30 19:07:26.615469 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:26.612756    5820 scope.go:122] "RemoveContainer" containerID="119c611b32b963acf4f8f965a2dfb4526d0745ef038d649615175a78978c39a0"
Oct 30 19:07:26.617534 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:07:26.617502    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"119c611b32b963acf4f8f965a2dfb4526d0745ef038d649615175a78978c39a0\": container with ID starting with 119c611b32b963acf4f8f965a2dfb4526d0745ef038d649615175a78978c39a0 not found: ID does not exist" containerID="119c611b32b963acf4f8f965a2dfb4526d0745ef038d649615175a78978c39a0"
Oct 30 19:07:26.617661 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:26.617544    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"119c611b32b963acf4f8f965a2dfb4526d0745ef038d649615175a78978c39a0"} err="failed to get container status \"119c611b32b963acf4f8f965a2dfb4526d0745ef038d649615175a78978c39a0\": rpc error: code = NotFound desc = could not find container \"119c611b32b963acf4f8f965a2dfb4526d0745ef038d649615175a78978c39a0\": container with ID starting with 119c611b32b963acf4f8f965a2dfb4526d0745ef038d649615175a78978c39a0 not found: ID does not exist"
Oct 30 19:07:26.765545 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:26.765510    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"]
Oct 30 19:07:26.814586 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:26.814558    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fee9f0-74bd9fc86-6fcb4"]
Oct 30 19:07:27.159375 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:27.159336    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:27.437107 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:27.437018    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e20e1010-00a5-490a-8aaa-93afaebe7932" path="/var/lib/kubelet/pods/e20e1010-00a5-490a-8aaa-93afaebe7932/volumes"
Oct 30 19:07:27.471579 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:27.471542    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:27.695067 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:27.694984    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:28.202291 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:28.202254    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:31.885199 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:31.885147    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:32.254048 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:32.253942    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:32.437784 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:32.437744    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:32.698114 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:32.698013    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:34.730092 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:34.730053    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:34.913061 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:34.913022    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:35.115591 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:35.115546    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:36.821453 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:36.821417    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:37.310163 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.310130    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"]
Oct 30 19:07:37.310842 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.310821    5820 cpu_manager.go:414] "containerMap: removing container" podUID="92af8465-a779-4904-b08c-aea834e86824" containerName="payload-processing"
Oct 30 19:07:37.310991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.310845    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92af8465-a779-4904-b08c-aea834e86824" containerName="payload-processing"
Oct 30 19:07:37.310991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.310899    5820 cpu_manager.go:414] "containerMap: removing container" podUID="fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d" containerName="payload-pre-processing"
Oct 30 19:07:37.310991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.310910    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d" containerName="payload-pre-processing"
Oct 30 19:07:37.310991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.310933    5820 cpu_manager.go:414] "containerMap: removing container" podUID="e20e1010-00a5-490a-8aaa-93afaebe7932" containerName="maas-api"
Oct 30 19:07:37.310991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.310940    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e20e1010-00a5-490a-8aaa-93afaebe7932" containerName="maas-api"
Oct 30 19:07:37.310991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.310971    5820 cpu_manager.go:414] "containerMap: removing container" podUID="db22814d-f854-4959-8f90-3c0e667ab59b" containerName="payload-processing"
Oct 30 19:07:37.310991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.310978    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db22814d-f854-4959-8f90-3c0e667ab59b" containerName="payload-processing"
Oct 30 19:07:37.310991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.310989    5820 cpu_manager.go:414] "containerMap: removing container" podUID="5fca5c67-64a5-44f6-9fb1-59d47874e39f" containerName="payload-processing"
Oct 30 19:07:37.311407 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.310996    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5fca5c67-64a5-44f6-9fb1-59d47874e39f" containerName="payload-processing"
Oct 30 19:07:37.311407 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.311019    5820 cpu_manager.go:414] "containerMap: removing container" podUID="f0408420-508d-475e-a6cb-85e9dfc06d4b" containerName="maas-api"
Oct 30 19:07:37.311407 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.311026    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0408420-508d-475e-a6cb-85e9dfc06d4b" containerName="maas-api"
Oct 30 19:07:37.311407 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.311205    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe43b403-5fe6-4f1b-a2fc-9bfde561ca7d" containerName="payload-pre-processing"
Oct 30 19:07:37.311407 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.311220    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="e20e1010-00a5-490a-8aaa-93afaebe7932" containerName="maas-api"
Oct 30 19:07:37.311407 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.311234    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="db22814d-f854-4959-8f90-3c0e667ab59b" containerName="payload-processing"
Oct 30 19:07:37.311407 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.311244    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="5fca5c67-64a5-44f6-9fb1-59d47874e39f" containerName="payload-processing"
Oct 30 19:07:37.311407 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.311256    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0408420-508d-475e-a6cb-85e9dfc06d4b" containerName="maas-api"
Oct 30 19:07:37.311407 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.311268    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="92af8465-a779-4904-b08c-aea834e86824" containerName="payload-processing"
Oct 30 19:07:37.315072 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.315051    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.409975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.409917    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-83ab2c-openshift-default-dockercfg-l5mzm\""
Oct 30 19:07:37.458988 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.451665    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:37.458988 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.455881    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"]
Oct 30 19:07:37.491247 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.491123    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-istio-data\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.491619 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.491590    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-workload-certs\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.491737 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.491658    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9384e565-f55a-45e3-bd08-b86c27972b91-istio-token\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.491737 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.491685    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-istio-envoy\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.491737 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.491719    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-workload-socket\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.491898 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.491804    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9384e565-f55a-45e3-bd08-b86c27972b91-istiod-ca-cert\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.491898 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.491852    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9384e565-f55a-45e3-bd08-b86c27972b91-istio-podinfo\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.492109 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.491941    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-credential-socket\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.492109 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.492018    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glcdp\" (UniqueName: \"kubernetes.io/projected/9384e565-f55a-45e3-bd08-b86c27972b91-kube-api-access-glcdp\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.596821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.596277    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-istio-data\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.596821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.596400    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-workload-certs\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.596821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.596451    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9384e565-f55a-45e3-bd08-b86c27972b91-istio-token\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.596821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.596477    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-istio-envoy\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.596821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.596517    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-workload-socket\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.596821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.596604    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9384e565-f55a-45e3-bd08-b86c27972b91-istiod-ca-cert\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.596821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.596651    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9384e565-f55a-45e3-bd08-b86c27972b91-istio-podinfo\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.596821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.596732    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-credential-socket\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.596821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.596763    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glcdp\" (UniqueName: \"kubernetes.io/projected/9384e565-f55a-45e3-bd08-b86c27972b91-kube-api-access-glcdp\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.597592 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.597564    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-istio-data\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.597806 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.597787    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-workload-certs\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.614208 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.614172    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9384e565-f55a-45e3-bd08-b86c27972b91-istiod-ca-cert\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.614557 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.614538    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-workload-socket\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.625239 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.625214    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9384e565-f55a-45e3-bd08-b86c27972b91-istio-podinfo\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.625762 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.625744    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-credential-socket\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.627647 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.627626    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-istio-envoy\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.639297 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.639269    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:37.658375 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.658353    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glcdp\" (UniqueName: \"kubernetes.io/projected/9384e565-f55a-45e3-bd08-b86c27972b91-kube-api-access-glcdp\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.658863 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.658840    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9384e565-f55a-45e3-bd08-b86c27972b91-istio-token\") pod \"e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") " pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:37.908253 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.908073    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:37.929273 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:37.929239    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:38.205378 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:07:38.205296    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:07:38.365808 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:38.365779    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"]
Oct 30 19:07:38.367716 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:07:38.367688    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9384e565_f55a_45e3_bd08_b86c27972b91.slice/crio-00f227d2c431cd10c16faf0e4b40586791ca165f2df176c5e5d1596a5c03c06c WatchSource:0}: Error finding container 00f227d2c431cd10c16faf0e4b40586791ca165f2df176c5e5d1596a5c03c06c: Status 404 returned error can't find the container with id 00f227d2c431cd10c16faf0e4b40586791ca165f2df176c5e5d1596a5c03c06c
Oct 30 19:07:38.370249 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:38.370214    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:07:38.370318 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:38.370287    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:07:38.370362 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:38.370322    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:07:38.685524 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:38.685489    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch" event={"ID":"9384e565-f55a-45e3-bd08-b86c27972b91","Type":"ContainerStarted","Data":"2573b5fca0645a9b31cdfd7306fb8d45beb95427a44e566994424f4e784547f9"}
Oct 30 19:07:38.685814 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:38.685558    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch" event={"ID":"9384e565-f55a-45e3-bd08-b86c27972b91","Type":"ContainerStarted","Data":"00f227d2c431cd10c16faf0e4b40586791ca165f2df176c5e5d1596a5c03c06c"}
Oct 30 19:07:38.807277 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:38.807215    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch" podStartSLOduration=1.8071976140000001 podStartE2EDuration="1.807197614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:38.80649113 +0000 UTC m=+1979.518239828" watchObservedRunningTime="2026-10-30 19:07:38.807197614 +0000 UTC m=+1979.518946313"
Oct 30 19:07:38.939701 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:38.937835    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:39.022258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:39.022223    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:39.336201 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:39.336167    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk"]
Oct 30 19:07:39.449370 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:39.449309    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586"]
Oct 30 19:07:39.741060 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:39.740941    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:39.932837 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:39.932784    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:39.937513 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:39.937487    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:40.164612 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:40.164571    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:40.707618 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:40.707586    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:40.708552 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:40.708531    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:07:41.862759 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:41.862709    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:42.237377 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:42.237288    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:42.427576 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:42.427533    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:42.625975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:42.622501    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:44.337359 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.337315    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk" podUID="1ec992e5-132c-4d80-9450-8021a3a51e0d" containerName="payload-processing" containerID="cri-o://bf16c4ee371e3f724e847ed9b97b21df67429b77af59c4be48b6215c194dbe66" gracePeriod=25
Oct 30 19:07:44.449982 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.449915    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586" podUID="177456c2-56a4-4d9b-839e-56287b82db19" containerName="payload-pre-processing" containerID="cri-o://7b700f0ee723b402386e9889a0abdf5b4e24ed4304b13148291d100956e07dbd" gracePeriod=25
Oct 30 19:07:44.607709 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.607631    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk"
Oct 30 19:07:44.660226 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.660196    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1ec992e5-132c-4d80-9450-8021a3a51e0d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ec992e5-132c-4d80-9450-8021a3a51e0d-plugins-config-volume\") pod \"1ec992e5-132c-4d80-9450-8021a3a51e0d\" (UID: \"1ec992e5-132c-4d80-9450-8021a3a51e0d\") "
Oct 30 19:07:44.660424 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.660248    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ec992e5-132c-4d80-9450-8021a3a51e0d-kube-api-access-h94cs\" (UniqueName: \"kubernetes.io/projected/1ec992e5-132c-4d80-9450-8021a3a51e0d-kube-api-access-h94cs\") pod \"1ec992e5-132c-4d80-9450-8021a3a51e0d\" (UID: \"1ec992e5-132c-4d80-9450-8021a3a51e0d\") "
Oct 30 19:07:44.660626 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.660602    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ec992e5-132c-4d80-9450-8021a3a51e0d-plugins-config-volume" pod "1ec992e5-132c-4d80-9450-8021a3a51e0d" (UID: "1ec992e5-132c-4d80-9450-8021a3a51e0d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:07:44.663083 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.663054    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ec992e5-132c-4d80-9450-8021a3a51e0d-kube-api-access-h94cs" pod "1ec992e5-132c-4d80-9450-8021a3a51e0d" (UID: "1ec992e5-132c-4d80-9450-8021a3a51e0d"). InnerVolumeSpecName "kube-api-access-h94cs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:44.719795 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.719769    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586"
Oct 30 19:07:44.761119 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.761090    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/177456c2-56a4-4d9b-839e-56287b82db19-kube-api-access-pggjw\" (UniqueName: \"kubernetes.io/projected/177456c2-56a4-4d9b-839e-56287b82db19-kube-api-access-pggjw\") pod \"177456c2-56a4-4d9b-839e-56287b82db19\" (UID: \"177456c2-56a4-4d9b-839e-56287b82db19\") "
Oct 30 19:07:44.761317 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.761164    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/177456c2-56a4-4d9b-839e-56287b82db19-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/177456c2-56a4-4d9b-839e-56287b82db19-plugins-config-volume\") pod \"177456c2-56a4-4d9b-839e-56287b82db19\" (UID: \"177456c2-56a4-4d9b-839e-56287b82db19\") "
Oct 30 19:07:44.761624 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.761598    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/177456c2-56a4-4d9b-839e-56287b82db19-plugins-config-volume" pod "177456c2-56a4-4d9b-839e-56287b82db19" (UID: "177456c2-56a4-4d9b-839e-56287b82db19"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:07:44.761736 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.761668    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ec992e5-132c-4d80-9450-8021a3a51e0d-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:44.761736 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.761690    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h94cs\" (UniqueName: \"kubernetes.io/projected/1ec992e5-132c-4d80-9450-8021a3a51e0d-kube-api-access-h94cs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:44.763481 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.763459    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/177456c2-56a4-4d9b-839e-56287b82db19-kube-api-access-pggjw" pod "177456c2-56a4-4d9b-839e-56287b82db19" (UID: "177456c2-56a4-4d9b-839e-56287b82db19"). InnerVolumeSpecName "kube-api-access-pggjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:44.797151 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.797117    5820 generic.go:356] "Generic (PLEG): container finished" podID="177456c2-56a4-4d9b-839e-56287b82db19" containerID="7b700f0ee723b402386e9889a0abdf5b4e24ed4304b13148291d100956e07dbd" exitCode=0
Oct 30 19:07:44.797332 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.797168    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586"
Oct 30 19:07:44.797332 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.797198    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586" event={"ID":"177456c2-56a4-4d9b-839e-56287b82db19","Type":"ContainerDied","Data":"7b700f0ee723b402386e9889a0abdf5b4e24ed4304b13148291d100956e07dbd"}
Oct 30 19:07:44.797332 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.797265    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586" event={"ID":"177456c2-56a4-4d9b-839e-56287b82db19","Type":"ContainerDied","Data":"e1484d9193ac7c8ffe1547078c5cf1ae341305768bf8b285d9a106a724a49931"}
Oct 30 19:07:44.797332 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.797295    5820 scope.go:122] "RemoveContainer" containerID="7b700f0ee723b402386e9889a0abdf5b4e24ed4304b13148291d100956e07dbd"
Oct 30 19:07:44.798876 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.798850    5820 generic.go:356] "Generic (PLEG): container finished" podID="1ec992e5-132c-4d80-9450-8021a3a51e0d" containerID="bf16c4ee371e3f724e847ed9b97b21df67429b77af59c4be48b6215c194dbe66" exitCode=0
Oct 30 19:07:44.799013 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.798982    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk" event={"ID":"1ec992e5-132c-4d80-9450-8021a3a51e0d","Type":"ContainerDied","Data":"bf16c4ee371e3f724e847ed9b97b21df67429b77af59c4be48b6215c194dbe66"}
Oct 30 19:07:44.799069 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.799032    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk" event={"ID":"1ec992e5-132c-4d80-9450-8021a3a51e0d","Type":"ContainerDied","Data":"691e6623a3879f88b7adce339ccf59498691a2d762b95ef83f4f8f5f2d48e627"}
Oct 30 19:07:44.799121 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.799103    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk"
Oct 30 19:07:44.809152 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.809135    5820 scope.go:122] "RemoveContainer" containerID="7b700f0ee723b402386e9889a0abdf5b4e24ed4304b13148291d100956e07dbd"
Oct 30 19:07:44.809422 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:07:44.809402    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b700f0ee723b402386e9889a0abdf5b4e24ed4304b13148291d100956e07dbd\": container with ID starting with 7b700f0ee723b402386e9889a0abdf5b4e24ed4304b13148291d100956e07dbd not found: ID does not exist" containerID="7b700f0ee723b402386e9889a0abdf5b4e24ed4304b13148291d100956e07dbd"
Oct 30 19:07:44.809525 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.809431    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b700f0ee723b402386e9889a0abdf5b4e24ed4304b13148291d100956e07dbd"} err="failed to get container status \"7b700f0ee723b402386e9889a0abdf5b4e24ed4304b13148291d100956e07dbd\": rpc error: code = NotFound desc = could not find container \"7b700f0ee723b402386e9889a0abdf5b4e24ed4304b13148291d100956e07dbd\": container with ID starting with 7b700f0ee723b402386e9889a0abdf5b4e24ed4304b13148291d100956e07dbd not found: ID does not exist"
Oct 30 19:07:44.809525 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.809452    5820 scope.go:122] "RemoveContainer" containerID="bf16c4ee371e3f724e847ed9b97b21df67429b77af59c4be48b6215c194dbe66"
Oct 30 19:07:44.818430 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.818401    5820 scope.go:122] "RemoveContainer" containerID="bf16c4ee371e3f724e847ed9b97b21df67429b77af59c4be48b6215c194dbe66"
Oct 30 19:07:44.818682 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:07:44.818661    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf16c4ee371e3f724e847ed9b97b21df67429b77af59c4be48b6215c194dbe66\": container with ID starting with bf16c4ee371e3f724e847ed9b97b21df67429b77af59c4be48b6215c194dbe66 not found: ID does not exist" containerID="bf16c4ee371e3f724e847ed9b97b21df67429b77af59c4be48b6215c194dbe66"
Oct 30 19:07:44.818750 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.818688    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf16c4ee371e3f724e847ed9b97b21df67429b77af59c4be48b6215c194dbe66"} err="failed to get container status \"bf16c4ee371e3f724e847ed9b97b21df67429b77af59c4be48b6215c194dbe66\": rpc error: code = NotFound desc = could not find container \"bf16c4ee371e3f724e847ed9b97b21df67429b77af59c4be48b6215c194dbe66\": container with ID starting with bf16c4ee371e3f724e847ed9b97b21df67429b77af59c4be48b6215c194dbe66 not found: ID does not exist"
Oct 30 19:07:44.863235 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.863145    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pggjw\" (UniqueName: \"kubernetes.io/projected/177456c2-56a4-4d9b-839e-56287b82db19-kube-api-access-pggjw\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:44.863235 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.863175    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/177456c2-56a4-4d9b-839e-56287b82db19-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:44.962325 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.962290    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk"]
Oct 30 19:07:44.967724 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:44.967696    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-842bbd-5674cd6bd6-rl7kk"]
Oct 30 19:07:45.080865 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:45.080826    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586"]
Oct 30 19:07:45.089505 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:45.089462    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-842bbd-66d99fc9b7-mt586"]
Oct 30 19:07:45.338027 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:45.336145    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:45.437510 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:45.437480    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="177456c2-56a4-4d9b-839e-56287b82db19" path="/var/lib/kubelet/pods/177456c2-56a4-4d9b-839e-56287b82db19/volumes"
Oct 30 19:07:45.437895 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:45.437882    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1ec992e5-132c-4d80-9450-8021a3a51e0d" path="/var/lib/kubelet/pods/1ec992e5-132c-4d80-9450-8021a3a51e0d/volumes"
Oct 30 19:07:48.517264 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:48.517174    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:48.694365 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:48.694322    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:48.900218 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:48.900184    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8"]
Oct 30 19:07:48.900795 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:48.900780    5820 cpu_manager.go:414] "containerMap: removing container" podUID="177456c2-56a4-4d9b-839e-56287b82db19" containerName="payload-pre-processing"
Oct 30 19:07:48.900879 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:48.900796    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="177456c2-56a4-4d9b-839e-56287b82db19" containerName="payload-pre-processing"
Oct 30 19:07:48.900879 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:48.900847    5820 cpu_manager.go:414] "containerMap: removing container" podUID="1ec992e5-132c-4d80-9450-8021a3a51e0d" containerName="payload-processing"
Oct 30 19:07:48.900879 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:48.900852    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ec992e5-132c-4d80-9450-8021a3a51e0d" containerName="payload-processing"
Oct 30 19:07:48.901006 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:48.900941    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ec992e5-132c-4d80-9450-8021a3a51e0d" containerName="payload-processing"
Oct 30 19:07:48.901006 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:48.900969    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="177456c2-56a4-4d9b-839e-56287b82db19" containerName="payload-pre-processing"
Oct 30 19:07:48.905736 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:48.905719    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8"
Oct 30 19:07:48.929132 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:48.929107    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbfhz\" (UniqueName: \"kubernetes.io/projected/3128fea4-f055-4fde-99e5-e71a21abe185-kube-api-access-gbfhz\") pod \"payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8\" (UID: \"3128fea4-f055-4fde-99e5-e71a21abe185\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8"
Oct 30 19:07:48.929274 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:48.929252    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3128fea4-f055-4fde-99e5-e71a21abe185-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8\" (UID: \"3128fea4-f055-4fde-99e5-e71a21abe185\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8"
Oct 30 19:07:48.961571 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:48.961533    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8"]
Oct 30 19:07:49.058978 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.038851    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbfhz\" (UniqueName: \"kubernetes.io/projected/3128fea4-f055-4fde-99e5-e71a21abe185-kube-api-access-gbfhz\") pod \"payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8\" (UID: \"3128fea4-f055-4fde-99e5-e71a21abe185\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8"
Oct 30 19:07:49.077043 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.077011    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3128fea4-f055-4fde-99e5-e71a21abe185-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8\" (UID: \"3128fea4-f055-4fde-99e5-e71a21abe185\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8"
Oct 30 19:07:49.085774 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.045689    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-a-83ab2c-dockercfg-lx8m4\""
Oct 30 19:07:49.086023 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.045568    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-a-83ab2c\""
Oct 30 19:07:49.086132 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.078175    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3128fea4-f055-4fde-99e5-e71a21abe185-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8\" (UID: \"3128fea4-f055-4fde-99e5-e71a21abe185\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8"
Oct 30 19:07:49.139066 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.126592    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbfhz\" (UniqueName: \"kubernetes.io/projected/3128fea4-f055-4fde-99e5-e71a21abe185-kube-api-access-gbfhz\") pod \"payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8\" (UID: \"3128fea4-f055-4fde-99e5-e71a21abe185\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8"
Oct 30 19:07:49.139066 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.133830    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx"]
Oct 30 19:07:49.142545 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.142516    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx"]
Oct 30 19:07:49.142762 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.142625    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx"
Oct 30 19:07:49.186446 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.186416    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f94tx\" (UniqueName: \"kubernetes.io/projected/54b02e73-6065-40b6-b0cb-bc2136f9b83e-kube-api-access-f94tx\") pod \"payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx\" (UID: \"54b02e73-6065-40b6-b0cb-bc2136f9b83e\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx"
Oct 30 19:07:49.186601 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.186471    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54b02e73-6065-40b6-b0cb-bc2136f9b83e-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx\" (UID: \"54b02e73-6065-40b6-b0cb-bc2136f9b83e\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx"
Oct 30 19:07:49.219802 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.219769    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8"
Oct 30 19:07:49.294061 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.293299    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54b02e73-6065-40b6-b0cb-bc2136f9b83e-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx\" (UID: \"54b02e73-6065-40b6-b0cb-bc2136f9b83e\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx"
Oct 30 19:07:49.294061 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.293603    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f94tx\" (UniqueName: \"kubernetes.io/projected/54b02e73-6065-40b6-b0cb-bc2136f9b83e-kube-api-access-f94tx\") pod \"payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx\" (UID: \"54b02e73-6065-40b6-b0cb-bc2136f9b83e\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx"
Oct 30 19:07:49.294406 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.294384    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54b02e73-6065-40b6-b0cb-bc2136f9b83e-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx\" (UID: \"54b02e73-6065-40b6-b0cb-bc2136f9b83e\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx"
Oct 30 19:07:49.356465 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.356428    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f94tx\" (UniqueName: \"kubernetes.io/projected/54b02e73-6065-40b6-b0cb-bc2136f9b83e-kube-api-access-f94tx\") pod \"payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx\" (UID: \"54b02e73-6065-40b6-b0cb-bc2136f9b83e\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx"
Oct 30 19:07:49.462092 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.462012    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx"
Oct 30 19:07:49.597259 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.597233    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8"]
Oct 30 19:07:49.599210 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:07:49.599183    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3128fea4_f055_4fde_99e5_e71a21abe185.slice/crio-a989b29f0a796b8eb2f46d2252ebadd8ff59ff2343587f93d96a286f4d0c8a8e WatchSource:0}: Error finding container a989b29f0a796b8eb2f46d2252ebadd8ff59ff2343587f93d96a286f4d0c8a8e: Status 404 returned error can't find the container with id a989b29f0a796b8eb2f46d2252ebadd8ff59ff2343587f93d96a286f4d0c8a8e
Oct 30 19:07:49.839819 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.839797    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx"]
Oct 30 19:07:49.842601 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:07:49.842563    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54b02e73_6065_40b6_b0cb_bc2136f9b83e.slice/crio-03bf8ae8d68f2e9eecee20393da0f7aee37e47efa072c4dd871dd830ef3940ec WatchSource:0}: Error finding container 03bf8ae8d68f2e9eecee20393da0f7aee37e47efa072c4dd871dd830ef3940ec: Status 404 returned error can't find the container with id 03bf8ae8d68f2e9eecee20393da0f7aee37e47efa072c4dd871dd830ef3940ec
Oct 30 19:07:49.846507 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.846474    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8" event={"ID":"3128fea4-f055-4fde-99e5-e71a21abe185","Type":"ContainerStarted","Data":"aa9ac7f8d2557555139624e78f9ac6f938f76d324f03045c4e9960a9cc803599"}
Oct 30 19:07:49.846619 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.846536    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8" event={"ID":"3128fea4-f055-4fde-99e5-e71a21abe185","Type":"ContainerStarted","Data":"a989b29f0a796b8eb2f46d2252ebadd8ff59ff2343587f93d96a286f4d0c8a8e"}
Oct 30 19:07:49.846707 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.846683    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8"
Oct 30 19:07:49.976155 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:49.976096    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8" podStartSLOduration=1.976081636 podStartE2EDuration="1.976081636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:49.9736127 +0000 UTC m=+1990.685361412" watchObservedRunningTime="2026-10-30 19:07:49.976081636 +0000 UTC m=+1990.687830332"
Oct 30 19:07:50.227186 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:50.227060    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:50.855082 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:50.855042    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx" event={"ID":"54b02e73-6065-40b6-b0cb-bc2136f9b83e","Type":"ContainerStarted","Data":"fc2b4fa99e76fdc395e1396c95507ce114e14e64e3626366a76bd04cce41568d"}
Oct 30 19:07:50.855509 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:50.855106    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx" event={"ID":"54b02e73-6065-40b6-b0cb-bc2136f9b83e","Type":"ContainerStarted","Data":"03bf8ae8d68f2e9eecee20393da0f7aee37e47efa072c4dd871dd830ef3940ec"}
Oct 30 19:07:50.855509 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:50.855193    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx"
Oct 30 19:07:50.973422 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:50.973370    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx" podStartSLOduration=1.973354784 podStartE2EDuration="1.973354784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:50.970212822 +0000 UTC m=+1991.681961520" watchObservedRunningTime="2026-10-30 19:07:50.973354784 +0000 UTC m=+1991.685103481"
Oct 30 19:07:55.145808 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:55.145768    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:57.642120 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:57.642081    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:59.515008 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:59.514944    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:59.906791 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:07:59.906748    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:00.104808 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.104757    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:00.519765 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.519725    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"]
Oct 30 19:08:00.527974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.526766    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.593842 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.593811    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-4ebf78-openshift-default-dockercfg-nsbrn\""
Oct 30 19:08:00.648972 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.643912    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:00.656991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.656166    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-workload-socket\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.656991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.656242    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-credential-socket\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.656991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.656267    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-data\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.656991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.656294    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-podinfo\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.656991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.656326    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-envoy\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.656991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.656392    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99f1797d-6313-4c12-a997-2c70b1c70d92-istiod-ca-cert\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.656991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.656477    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-token\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.656991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.656508    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmcps\" (UniqueName: \"kubernetes.io/projected/99f1797d-6313-4c12-a997-2c70b1c70d92-kube-api-access-wmcps\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.656991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.656575    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-workload-certs\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.692972 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.686601    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"]
Oct 30 19:08:00.763620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.761025    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-token\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.763620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.761074    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmcps\" (UniqueName: \"kubernetes.io/projected/99f1797d-6313-4c12-a997-2c70b1c70d92-kube-api-access-wmcps\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.763620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.761137    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-workload-certs\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.763620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.761251    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-workload-socket\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.763620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.761302    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-credential-socket\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.763620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.761323    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-data\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.763620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.761349    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-podinfo\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.763620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.761382    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-envoy\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.763620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.761441    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99f1797d-6313-4c12-a997-2c70b1c70d92-istiod-ca-cert\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.763620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.762155    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-workload-socket\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.763620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.762223    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99f1797d-6313-4c12-a997-2c70b1c70d92-istiod-ca-cert\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.763620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.762446    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-credential-socket\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.763620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.762646    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-data\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.763620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.763402    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-workload-certs\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.767986 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.766201    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-envoy\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.787382 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.784317    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-podinfo\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.824492 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.824450    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmcps\" (UniqueName: \"kubernetes.io/projected/99f1797d-6313-4c12-a997-2c70b1c70d92-kube-api-access-wmcps\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.825377 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.825349    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-token\") pod \"e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") " pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:00.858619 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.858593    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8"
Oct 30 19:08:00.953529 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:00.953464    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:01.028931 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:01.028900    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:01.399288 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:01.399263    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"]
Oct 30 19:08:01.400227 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:08:01.400190    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99f1797d_6313_4c12_a997_2c70b1c70d92.slice/crio-726ad4d4af90264974df0399746d968914cebc88a107397f780f0b0ec1fe374e WatchSource:0}: Error finding container 726ad4d4af90264974df0399746d968914cebc88a107397f780f0b0ec1fe374e: Status 404 returned error can't find the container with id 726ad4d4af90264974df0399746d968914cebc88a107397f780f0b0ec1fe374e
Oct 30 19:08:01.402996 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:01.402918    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:08:01.403121 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:01.403034    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:08:01.403121 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:01.403075    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:08:01.865467 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:01.865443    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx"
Oct 30 19:08:01.926026 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:01.925992    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb" event={"ID":"99f1797d-6313-4c12-a997-2c70b1c70d92","Type":"ContainerStarted","Data":"8573b112d9779ec6306a4dcc10578c21f9b0f2a0c7f3bff6aff99a758b1f5d22"}
Oct 30 19:08:01.926199 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:01.926048    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb" event={"ID":"99f1797d-6313-4c12-a997-2c70b1c70d92","Type":"ContainerStarted","Data":"726ad4d4af90264974df0399746d968914cebc88a107397f780f0b0ec1fe374e"}
Oct 30 19:08:01.963767 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:01.963736    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:02.104038 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:02.103908    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb" podStartSLOduration=2.103892464 podStartE2EDuration="2.103892464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:02.102029693 +0000 UTC m=+2002.813778417" watchObservedRunningTime="2026-10-30 19:08:02.103892464 +0000 UTC m=+2002.815641160"
Oct 30 19:08:02.968982 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:02.968927    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:03.940166 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:03.940128    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:03.941181 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:03.941160    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:08:05.025262 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:05.025226    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:05.261520 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:05.261477    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:08.208407 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:08:08.208361    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:08:08.281069 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:08.281036    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:08.508175 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:08.508082    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:09.251766 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.251734    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"]
Oct 30 19:08:09.252236 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.251922    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv" podUID="47e07161-1cfe-4075-84fb-52a095c05533" containerName="maas-api" containerID="cri-o://b1067935dff729362446d98e0407337c2752bdebed682fa24e9f8fcd131e437e" gracePeriod=30
Oct 30 19:08:09.392803 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.392768    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:09.564784 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.564761    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"
Oct 30 19:08:09.752077 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.752037    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/47e07161-1cfe-4075-84fb-52a095c05533-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47e07161-1cfe-4075-84fb-52a095c05533-maas-api-tls\") pod \"47e07161-1cfe-4075-84fb-52a095c05533\" (UID: \"47e07161-1cfe-4075-84fb-52a095c05533\") "
Oct 30 19:08:09.758189 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.754459    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47e07161-1cfe-4075-84fb-52a095c05533-kube-api-access-5cgzn\" (UniqueName: \"kubernetes.io/projected/47e07161-1cfe-4075-84fb-52a095c05533-kube-api-access-5cgzn\") pod \"47e07161-1cfe-4075-84fb-52a095c05533\" (UID: \"47e07161-1cfe-4075-84fb-52a095c05533\") "
Oct 30 19:08:09.758189 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.754505    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/47e07161-1cfe-4075-84fb-52a095c05533-metrics-certs\" (UniqueName: \"kubernetes.io/secret/47e07161-1cfe-4075-84fb-52a095c05533-metrics-certs\") pod \"47e07161-1cfe-4075-84fb-52a095c05533\" (UID: \"47e07161-1cfe-4075-84fb-52a095c05533\") "
Oct 30 19:08:09.758189 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.754632    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/47e07161-1cfe-4075-84fb-52a095c05533-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47e07161-1cfe-4075-84fb-52a095c05533-openshift-service-ca\") pod \"47e07161-1cfe-4075-84fb-52a095c05533\" (UID: \"47e07161-1cfe-4075-84fb-52a095c05533\") "
Oct 30 19:08:09.758189 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.755509    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47e07161-1cfe-4075-84fb-52a095c05533-openshift-service-ca" pod "47e07161-1cfe-4075-84fb-52a095c05533" (UID: "47e07161-1cfe-4075-84fb-52a095c05533"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:08:09.758189 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.758087    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47e07161-1cfe-4075-84fb-52a095c05533-kube-api-access-5cgzn" pod "47e07161-1cfe-4075-84fb-52a095c05533" (UID: "47e07161-1cfe-4075-84fb-52a095c05533"). InnerVolumeSpecName "kube-api-access-5cgzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:09.760636 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.760567    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47e07161-1cfe-4075-84fb-52a095c05533-metrics-certs" pod "47e07161-1cfe-4075-84fb-52a095c05533" (UID: "47e07161-1cfe-4075-84fb-52a095c05533"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:08:09.761194 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.761159    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47e07161-1cfe-4075-84fb-52a095c05533-maas-api-tls" pod "47e07161-1cfe-4075-84fb-52a095c05533" (UID: "47e07161-1cfe-4075-84fb-52a095c05533"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:08:09.857066 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.857031    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47e07161-1cfe-4075-84fb-52a095c05533-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:09.857066 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.857069    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cgzn\" (UniqueName: \"kubernetes.io/projected/47e07161-1cfe-4075-84fb-52a095c05533-kube-api-access-5cgzn\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:09.857320 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.857086    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47e07161-1cfe-4075-84fb-52a095c05533-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:09.857320 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.857099    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47e07161-1cfe-4075-84fb-52a095c05533-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:09.945728 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.945697    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:09.981288 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.981255    5820 generic.go:356] "Generic (PLEG): container finished" podID="47e07161-1cfe-4075-84fb-52a095c05533" containerID="b1067935dff729362446d98e0407337c2752bdebed682fa24e9f8fcd131e437e" exitCode=0
Oct 30 19:08:09.981450 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.981318    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"
Oct 30 19:08:09.981450 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.981328    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv" event={"ID":"47e07161-1cfe-4075-84fb-52a095c05533","Type":"ContainerDied","Data":"b1067935dff729362446d98e0407337c2752bdebed682fa24e9f8fcd131e437e"}
Oct 30 19:08:09.981450 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.981389    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv" event={"ID":"47e07161-1cfe-4075-84fb-52a095c05533","Type":"ContainerDied","Data":"855671f34dbf6f8bf6305a86a2d45d981bc904e17c3a465ac041bea058bd90ed"}
Oct 30 19:08:09.981450 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.981411    5820 scope.go:122] "RemoveContainer" containerID="b1067935dff729362446d98e0407337c2752bdebed682fa24e9f8fcd131e437e"
Oct 30 19:08:09.992874 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.992853    5820 scope.go:122] "RemoveContainer" containerID="b1067935dff729362446d98e0407337c2752bdebed682fa24e9f8fcd131e437e"
Oct 30 19:08:09.993199 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:08:09.993178    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1067935dff729362446d98e0407337c2752bdebed682fa24e9f8fcd131e437e\": container with ID starting with b1067935dff729362446d98e0407337c2752bdebed682fa24e9f8fcd131e437e not found: ID does not exist" containerID="b1067935dff729362446d98e0407337c2752bdebed682fa24e9f8fcd131e437e"
Oct 30 19:08:09.993239 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:09.993208    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1067935dff729362446d98e0407337c2752bdebed682fa24e9f8fcd131e437e"} err="failed to get container status \"b1067935dff729362446d98e0407337c2752bdebed682fa24e9f8fcd131e437e\": rpc error: code = NotFound desc = could not find container \"b1067935dff729362446d98e0407337c2752bdebed682fa24e9f8fcd131e437e\": container with ID starting with b1067935dff729362446d98e0407337c2752bdebed682fa24e9f8fcd131e437e not found: ID does not exist"
Oct 30 19:08:10.150382 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:10.150288    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"]
Oct 30 19:08:10.152783 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:10.152758    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a23af5-566f777d46-sfqrv"]
Oct 30 19:08:10.173431 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:10.173402    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:11.015188 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:11.015157    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:11.304675 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:11.304638    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:11.437692 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:11.437658    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="47e07161-1cfe-4075-84fb-52a095c05533" path="/var/lib/kubelet/pods/47e07161-1cfe-4075-84fb-52a095c05533/volumes"
Oct 30 19:08:11.480901 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:11.480870    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:11.950606 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:11.950567    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:13.899425 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:13.899386    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:16.850977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:16.848943    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:18.626279 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:18.626241    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:18.811236 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:18.811202    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:18.967546 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:18.967467    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"]
Oct 30 19:08:18.967722 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:18.967685    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n" podUID="8697cc5c-98e3-418b-9d0a-fd6930ce317e" containerName="istio-proxy" containerID="cri-o://84051cc564143f2a0f9f0ee28eda83122ad8a7dfd4c88712c9b5c4846fbfbc47" gracePeriod=30
Oct 30 19:08:18.998339 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:18.998306    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:20.709246 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:20.709206    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:21.287128 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:21.287093    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:24.174971 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.170181    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:24.192782 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.192429    5820 generic.go:356] "Generic (PLEG): container finished" podID="8697cc5c-98e3-418b-9d0a-fd6930ce317e" containerID="84051cc564143f2a0f9f0ee28eda83122ad8a7dfd4c88712c9b5c4846fbfbc47" exitCode=0
Oct 30 19:08:24.192782 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.192526    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n" event={"ID":"8697cc5c-98e3-418b-9d0a-fd6930ce317e","Type":"ContainerDied","Data":"84051cc564143f2a0f9f0ee28eda83122ad8a7dfd4c88712c9b5c4846fbfbc47"}
Oct 30 19:08:24.192782 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.192569    5820 scope.go:122] "RemoveContainer" containerID="e612c5d221069d4238d52d60e6d194e6fa8a4d138763330d9041265d56756db2"
Oct 30 19:08:24.290858 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.290835    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:08:24.369463 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.369426    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-podinfo\") pod \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") "
Oct 30 19:08:24.369660 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.369507    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8697cc5c-98e3-418b-9d0a-fd6930ce317e-kube-api-access-bfm8s\" (UniqueName: \"kubernetes.io/projected/8697cc5c-98e3-418b-9d0a-fd6930ce317e-kube-api-access-bfm8s\") pod \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") "
Oct 30 19:08:24.369660 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.369563    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-data\") pod \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") "
Oct 30 19:08:24.369660 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.369613    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-envoy\") pod \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") "
Oct 30 19:08:24.369826 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.369660    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-credential-socket\") pod \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") "
Oct 30 19:08:24.369826 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.369698    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-token\" (UniqueName: \"kubernetes.io/projected/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-token\") pod \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") "
Oct 30 19:08:24.369826 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.369804    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-workload-certs\") pod \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") "
Oct 30 19:08:24.370015 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.369853    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istiod-ca-cert\") pod \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") "
Oct 30 19:08:24.370015 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.369895    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-workload-socket\") pod \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\" (UID: \"8697cc5c-98e3-418b-9d0a-fd6930ce317e\") "
Oct 30 19:08:24.370146 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.369849    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-data" pod "8697cc5c-98e3-418b-9d0a-fd6930ce317e" (UID: "8697cc5c-98e3-418b-9d0a-fd6930ce317e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:24.370208 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.370153    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-workload-certs" pod "8697cc5c-98e3-418b-9d0a-fd6930ce317e" (UID: "8697cc5c-98e3-418b-9d0a-fd6930ce317e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:24.370208 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.369942    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-credential-socket" pod "8697cc5c-98e3-418b-9d0a-fd6930ce317e" (UID: "8697cc5c-98e3-418b-9d0a-fd6930ce317e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:24.370363 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.370344    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-workload-socket" pod "8697cc5c-98e3-418b-9d0a-fd6930ce317e" (UID: "8697cc5c-98e3-418b-9d0a-fd6930ce317e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:24.370495 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.370474    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istiod-ca-cert" pod "8697cc5c-98e3-418b-9d0a-fd6930ce317e" (UID: "8697cc5c-98e3-418b-9d0a-fd6930ce317e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:08:24.370724 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.370706    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:24.370788 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.370731    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:24.370788 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.370747    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:24.370788 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.370762    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:24.370788 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.370776    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:24.371996 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.371975    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8697cc5c-98e3-418b-9d0a-fd6930ce317e-kube-api-access-bfm8s" pod "8697cc5c-98e3-418b-9d0a-fd6930ce317e" (UID: "8697cc5c-98e3-418b-9d0a-fd6930ce317e"). InnerVolumeSpecName "kube-api-access-bfm8s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:24.372091 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.372053    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-podinfo" pod "8697cc5c-98e3-418b-9d0a-fd6930ce317e" (UID: "8697cc5c-98e3-418b-9d0a-fd6930ce317e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:08:24.372491 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.372470    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-envoy" pod "8697cc5c-98e3-418b-9d0a-fd6930ce317e" (UID: "8697cc5c-98e3-418b-9d0a-fd6930ce317e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:24.372556 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.372489    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-token" pod "8697cc5c-98e3-418b-9d0a-fd6930ce317e" (UID: "8697cc5c-98e3-418b-9d0a-fd6930ce317e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:24.472044 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.472015    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:24.472044 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.472041    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:24.472044 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.472052    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8697cc5c-98e3-418b-9d0a-fd6930ce317e-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:24.472300 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:24.472064    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfm8s\" (UniqueName: \"kubernetes.io/projected/8697cc5c-98e3-418b-9d0a-fd6930ce317e-kube-api-access-bfm8s\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:25.200312 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:25.200280    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"
Oct 30 19:08:25.200796 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:25.200325    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n" event={"ID":"8697cc5c-98e3-418b-9d0a-fd6930ce317e","Type":"ContainerDied","Data":"78509beda30a55bbe454291d335485295c77be0713d98f8125d407a667859b75"}
Oct 30 19:08:25.200796 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:25.200397    5820 scope.go:122] "RemoveContainer" containerID="84051cc564143f2a0f9f0ee28eda83122ad8a7dfd4c88712c9b5c4846fbfbc47"
Oct 30 19:08:25.415965 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:25.415927    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"]
Oct 30 19:08:25.465559 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:25.465482    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-a23af5-openshift-default-7ff86c6d59-ljn4n"]
Oct 30 19:08:27.437420 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:27.437383    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8697cc5c-98e3-418b-9d0a-fd6930ce317e" path="/var/lib/kubelet/pods/8697cc5c-98e3-418b-9d0a-fd6930ce317e/volumes"
Oct 30 19:08:29.536973 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:29.536056    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:33.346729 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:33.346688    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:33.842023 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:33.841985    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:35.708324 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:35.708287    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7"]
Oct 30 19:08:35.709298 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:35.709277    5820 cpu_manager.go:414] "containerMap: removing container" podUID="8697cc5c-98e3-418b-9d0a-fd6930ce317e" containerName="istio-proxy"
Oct 30 19:08:35.709354 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:35.709303    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8697cc5c-98e3-418b-9d0a-fd6930ce317e" containerName="istio-proxy"
Oct 30 19:08:35.709354 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:35.709337    5820 cpu_manager.go:414] "containerMap: removing container" podUID="47e07161-1cfe-4075-84fb-52a095c05533" containerName="maas-api"
Oct 30 19:08:35.709354 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:35.709345    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47e07161-1cfe-4075-84fb-52a095c05533" containerName="maas-api"
Oct 30 19:08:35.709453 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:35.709409    5820 cpu_manager.go:414] "containerMap: removing container" podUID="8697cc5c-98e3-418b-9d0a-fd6930ce317e" containerName="istio-proxy"
Oct 30 19:08:35.709453 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:35.709417    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8697cc5c-98e3-418b-9d0a-fd6930ce317e" containerName="istio-proxy"
Oct 30 19:08:35.709564 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:35.709551    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="47e07161-1cfe-4075-84fb-52a095c05533" containerName="maas-api"
Oct 30 19:08:35.709600 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:35.709581    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="8697cc5c-98e3-418b-9d0a-fd6930ce317e" containerName="istio-proxy"
Oct 30 19:08:35.713784 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:35.713760    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7"
Oct 30 19:08:35.765885 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:35.765846    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-0ee379\""
Oct 30 19:08:35.825971 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:35.818882    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gzbp\" (UniqueName: \"kubernetes.io/projected/19332f09-bec4-403a-9a0c-f7989f71bba7-kube-api-access-5gzbp\") pod \"payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7\" (UID: \"19332f09-bec4-403a-9a0c-f7989f71bba7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7"
Oct 30 19:08:35.825971 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:35.819073    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19332f09-bec4-403a-9a0c-f7989f71bba7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7\" (UID: \"19332f09-bec4-403a-9a0c-f7989f71bba7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7"
Oct 30 19:08:35.828684 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:35.828656    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-0ee379-dockercfg-4vpdm\""
Oct 30 19:08:35.833429 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:35.833401    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7"]
Oct 30 19:08:35.920186 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:35.920153    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19332f09-bec4-403a-9a0c-f7989f71bba7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7\" (UID: \"19332f09-bec4-403a-9a0c-f7989f71bba7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7"
Oct 30 19:08:35.920397 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:35.920281    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gzbp\" (UniqueName: \"kubernetes.io/projected/19332f09-bec4-403a-9a0c-f7989f71bba7-kube-api-access-5gzbp\") pod \"payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7\" (UID: \"19332f09-bec4-403a-9a0c-f7989f71bba7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7"
Oct 30 19:08:35.920923 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:35.920898    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19332f09-bec4-403a-9a0c-f7989f71bba7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7\" (UID: \"19332f09-bec4-403a-9a0c-f7989f71bba7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7"
Oct 30 19:08:35.981703 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:35.981628    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gzbp\" (UniqueName: \"kubernetes.io/projected/19332f09-bec4-403a-9a0c-f7989f71bba7-kube-api-access-5gzbp\") pod \"payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7\" (UID: \"19332f09-bec4-403a-9a0c-f7989f71bba7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7"
Oct 30 19:08:36.031708 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:36.031678    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7"
Oct 30 19:08:36.409032 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:36.409002    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7"]
Oct 30 19:08:36.411034 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:08:36.411002    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19332f09_bec4_403a_9a0c_f7989f71bba7.slice/crio-6dad8a3ef1e5b7b039a42ec4b590819e30dd778ca77a828766b5107a1460291d WatchSource:0}: Error finding container 6dad8a3ef1e5b7b039a42ec4b590819e30dd778ca77a828766b5107a1460291d: Status 404 returned error can't find the container with id 6dad8a3ef1e5b7b039a42ec4b590819e30dd778ca77a828766b5107a1460291d
Oct 30 19:08:36.631417 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:36.631381    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:37.306260 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:37.306222    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7" event={"ID":"19332f09-bec4-403a-9a0c-f7989f71bba7","Type":"ContainerStarted","Data":"9fe6ff9c887800cf7d248efcbc9f6bea3f672fe1a0d2912f409990192c60f568"}
Oct 30 19:08:37.306681 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:37.306308    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7" event={"ID":"19332f09-bec4-403a-9a0c-f7989f71bba7","Type":"ContainerStarted","Data":"6dad8a3ef1e5b7b039a42ec4b590819e30dd778ca77a828766b5107a1460291d"}
Oct 30 19:08:37.306681 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:37.306458    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7"
Oct 30 19:08:37.426277 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:37.426222    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7" podStartSLOduration=2.426209516 podStartE2EDuration="2.426209516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:37.422140532 +0000 UTC m=+2038.133889229" watchObservedRunningTime="2026-10-30 19:08:37.426209516 +0000 UTC m=+2038.137958213"
Oct 30 19:08:38.203633 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:08:38.203593    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:08:40.066195 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.066159    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:40.561726 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.561689    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"]
Oct 30 19:08:40.562622 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.562597    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="8697cc5c-98e3-418b-9d0a-fd6930ce317e" containerName="istio-proxy"
Oct 30 19:08:40.574165 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.574140    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.650012 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.635183    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-7f44c7-openshift-default-dockercfg-qwvpm\""
Oct 30 19:08:40.698139 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.691850    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-istio-envoy\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.699926 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.699831    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-credential-socket\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.707981 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.700276    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-istio-data\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.711808 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.708249    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-workload-certs\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.728014 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.715112    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt2dv\" (UniqueName: \"kubernetes.io/projected/223cde4d-b792-432f-80ca-fb0189521a71-kube-api-access-kt2dv\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.732845 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.732794    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/223cde4d-b792-432f-80ca-fb0189521a71-istiod-ca-cert\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.733271 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.733246    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-workload-socket\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.733689 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.733666    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/223cde4d-b792-432f-80ca-fb0189521a71-istio-podinfo\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.733895 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.733877    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/223cde4d-b792-432f-80ca-fb0189521a71-istio-token\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.815963 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.761990    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:40.822013 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.765421    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"]
Oct 30 19:08:40.835598 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.835558    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/223cde4d-b792-432f-80ca-fb0189521a71-istio-podinfo\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.835797 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.835615    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/223cde4d-b792-432f-80ca-fb0189521a71-istio-token\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.835797 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.835705    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-istio-envoy\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.835797 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.835737    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-credential-socket\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.835797 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.835773    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-istio-data\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.836085 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.835827    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-workload-certs\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.836085 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.835921    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt2dv\" (UniqueName: \"kubernetes.io/projected/223cde4d-b792-432f-80ca-fb0189521a71-kube-api-access-kt2dv\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.836085 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.835982    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/223cde4d-b792-432f-80ca-fb0189521a71-istiod-ca-cert\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.836085 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.836014    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-workload-socket\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.836452 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.836428    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-workload-certs\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.836654 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.836626    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-workload-socket\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.836772 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.836692    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-istio-data\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.836772 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.836719    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-credential-socket\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.837162 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.837134    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/223cde4d-b792-432f-80ca-fb0189521a71-istiod-ca-cert\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.839307 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.839283    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-istio-envoy\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.839455 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.839428    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/223cde4d-b792-432f-80ca-fb0189521a71-istio-podinfo\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.916242 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.916213    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt2dv\" (UniqueName: \"kubernetes.io/projected/223cde4d-b792-432f-80ca-fb0189521a71-kube-api-access-kt2dv\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.918267 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.918244    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/223cde4d-b792-432f-80ca-fb0189521a71-istio-token\") pod \"e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") " pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:40.935039 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:40.935015    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:41.178428 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:41.178337    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:41.370626 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:41.370590    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"]
Oct 30 19:08:41.370986 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:08:41.370938    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod223cde4d_b792_432f_80ca_fb0189521a71.slice/crio-abe9ba0dd60a33cbb3d59a0fdfe7e76256f4aa0950518a2f089fb44d8b1e355b WatchSource:0}: Error finding container abe9ba0dd60a33cbb3d59a0fdfe7e76256f4aa0950518a2f089fb44d8b1e355b: Status 404 returned error can't find the container with id abe9ba0dd60a33cbb3d59a0fdfe7e76256f4aa0950518a2f089fb44d8b1e355b
Oct 30 19:08:41.373438 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:41.373406    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:08:41.373516 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:41.373469    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:08:41.373516 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:41.373498    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:08:42.338715 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:42.338677    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t" event={"ID":"223cde4d-b792-432f-80ca-fb0189521a71","Type":"ContainerStarted","Data":"60d15b8bdd9eb7c0a539a8d15b12e2b971d4d6f31191180766b0eee8eed16b30"}
Oct 30 19:08:42.339147 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:42.338735    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t" event={"ID":"223cde4d-b792-432f-80ca-fb0189521a71","Type":"ContainerStarted","Data":"abe9ba0dd60a33cbb3d59a0fdfe7e76256f4aa0950518a2f089fb44d8b1e355b"}
Oct 30 19:08:42.559226 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:42.559167    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t" podStartSLOduration=2.55915449 podStartE2EDuration="2.55915449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:42.507348056 +0000 UTC m=+2043.219096754" watchObservedRunningTime="2026-10-30 19:08:42.55915449 +0000 UTC m=+2043.270903187"
Oct 30 19:08:42.848032 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:42.848000    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"]
Oct 30 19:08:42.852312 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:42.852295    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"
Oct 30 19:08:42.907546 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:42.907493    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-83ab2c-serving-cert\""
Oct 30 19:08:42.912125 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:42.912100    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"]
Oct 30 19:08:42.935540 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:42.935515    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:42.942734 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:42.942709    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:42.961842 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:42.961811    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-metrics-certs\") pod \"maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl\" (UID: \"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"
Oct 30 19:08:42.961996 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:42.961928    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkng6\" (UniqueName: \"kubernetes.io/projected/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-kube-api-access-vkng6\") pod \"maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl\" (UID: \"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"
Oct 30 19:08:42.962049 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:42.962002    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-maas-api-tls\") pod \"maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl\" (UID: \"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"
Oct 30 19:08:42.962084 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:42.962052    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl\" (UID: \"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"
Oct 30 19:08:43.063696 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:43.063662    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkng6\" (UniqueName: \"kubernetes.io/projected/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-kube-api-access-vkng6\") pod \"maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl\" (UID: \"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"
Oct 30 19:08:43.063884 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:43.063731    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-maas-api-tls\") pod \"maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl\" (UID: \"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"
Oct 30 19:08:43.063884 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:43.063780    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl\" (UID: \"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"
Oct 30 19:08:43.064030 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:08:43.063905    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-serving-cert: secret "maas-api-e2e-praxis-a-83ab2c-serving-cert" not found
Oct 30 19:08:43.064030 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:43.063941    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-metrics-certs\") pod \"maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl\" (UID: \"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"
Oct 30 19:08:43.064030 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:08:43.063995    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-maas-api-tls podName:8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462 nodeName:}" failed. No retries permitted until 2026-10-30 19:08:43.563972444 +0000 UTC m=+2044.275721136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-maas-api-tls") pod "maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl" (UID: "8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462") : secret "maas-api-e2e-praxis-a-83ab2c-serving-cert" not found
Oct 30 19:08:43.064624 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:43.064595    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl\" (UID: \"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"
Oct 30 19:08:43.066577 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:43.066551    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-metrics-certs\") pod \"maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl\" (UID: \"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"
Oct 30 19:08:43.077207 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:43.077178    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:43.130266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:43.130182    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkng6\" (UniqueName: \"kubernetes.io/projected/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-kube-api-access-vkng6\") pod \"maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl\" (UID: \"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"
Oct 30 19:08:43.301065 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:43.301026    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:43.344965 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:43.344917    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:43.346125 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:43.346108    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:08:43.583806 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:43.583772    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-maas-api-tls\") pod \"maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl\" (UID: \"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"
Oct 30 19:08:43.586603 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:43.586578    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-maas-api-tls\") pod \"maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl\" (UID: \"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"
Oct 30 19:08:43.767713 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:43.767676    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"
Oct 30 19:08:44.147828 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:44.147800    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"]
Oct 30 19:08:44.149785 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:08:44.149758    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ca42ea4_0b3c_424e_ba8e_7f6cd7e00462.slice/crio-dc88f059b09b9531ba41dcd6b450b14b6489f224a328a7bcb1200d2cb62eb3b9 WatchSource:0}: Error finding container dc88f059b09b9531ba41dcd6b450b14b6489f224a328a7bcb1200d2cb62eb3b9: Status 404 returned error can't find the container with id dc88f059b09b9531ba41dcd6b450b14b6489f224a328a7bcb1200d2cb62eb3b9
Oct 30 19:08:44.352585 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:44.352550    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl" event={"ID":"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462","Type":"ContainerStarted","Data":"dc88f059b09b9531ba41dcd6b450b14b6489f224a328a7bcb1200d2cb62eb3b9"}
Oct 30 19:08:45.360577 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:45.360540    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl" event={"ID":"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462","Type":"ContainerStarted","Data":"beb7b097b2e23f68199816bf1724dcef26e745603c9fc60c2cca98888295129c"}
Oct 30 19:08:45.361029 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:45.360860    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"
Oct 30 19:08:45.385633 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:45.385596    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:45.485784 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:45.485732    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl" podStartSLOduration=2.59264979 podStartE2EDuration="3.485718369s" totalImagesPullingTime="893.068579ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:45.48441078 +0000 UTC m=+2046.196159478" watchObservedRunningTime="2026-10-30 19:08:45.485718369 +0000 UTC m=+2046.197467093"
Oct 30 19:08:45.579597 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:45.579558    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:45.769512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:45.769367    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:48.314821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:48.314787    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7"
Oct 30 19:08:48.369786 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:48.369755    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:48.626119 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:48.626037    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:48.812503 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:48.812448    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:50.349456 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:50.349411    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:50.568707 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:50.568674    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:51.372440 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:51.372410    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"
Oct 30 19:08:53.574003 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:53.573969    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:57.983133 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:57.983083    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:58.644474 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:08:58.644441    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:03.943575 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:03.943534    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:08.207001 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:09:08.206936    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:09:09.055062 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:09.055019    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:09.262969 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:09.261554    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:09.440461 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:09.440374    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:13.728315 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:13.728276    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"]
Oct 30 19:09:13.728962 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:13.728536    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg" podUID="727362a7-2b8f-401d-afbd-71a8dcc2fd7f" containerName="maas-api" containerID="cri-o://f7ac85c163ab31b1a5a2771728303aaa3d1ef44caf225ee09e5b187e45f08d27" gracePeriod=30
Oct 30 19:09:13.889285 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:13.889252    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:14.073802 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:14.073773    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"
Oct 30 19:09:14.200969 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:14.199744    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-kube-api-access-g2nxq\" (UniqueName: \"kubernetes.io/projected/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-kube-api-access-g2nxq\") pod \"727362a7-2b8f-401d-afbd-71a8dcc2fd7f\" (UID: \"727362a7-2b8f-401d-afbd-71a8dcc2fd7f\") "
Oct 30 19:09:14.200969 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:14.199794    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-maas-api-tls\") pod \"727362a7-2b8f-401d-afbd-71a8dcc2fd7f\" (UID: \"727362a7-2b8f-401d-afbd-71a8dcc2fd7f\") "
Oct 30 19:09:14.200969 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:14.200008    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-metrics-certs\") pod \"727362a7-2b8f-401d-afbd-71a8dcc2fd7f\" (UID: \"727362a7-2b8f-401d-afbd-71a8dcc2fd7f\") "
Oct 30 19:09:14.200969 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:14.200048    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-openshift-service-ca\") pod \"727362a7-2b8f-401d-afbd-71a8dcc2fd7f\" (UID: \"727362a7-2b8f-401d-afbd-71a8dcc2fd7f\") "
Oct 30 19:09:14.201793 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:14.201485    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-openshift-service-ca" pod "727362a7-2b8f-401d-afbd-71a8dcc2fd7f" (UID: "727362a7-2b8f-401d-afbd-71a8dcc2fd7f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:09:14.236199 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:14.229909    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-kube-api-access-g2nxq" pod "727362a7-2b8f-401d-afbd-71a8dcc2fd7f" (UID: "727362a7-2b8f-401d-afbd-71a8dcc2fd7f"). InnerVolumeSpecName "kube-api-access-g2nxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:09:14.236199 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:14.231621    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-maas-api-tls" pod "727362a7-2b8f-401d-afbd-71a8dcc2fd7f" (UID: "727362a7-2b8f-401d-afbd-71a8dcc2fd7f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:09:14.237590 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:14.237533    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-metrics-certs" pod "727362a7-2b8f-401d-afbd-71a8dcc2fd7f" (UID: "727362a7-2b8f-401d-afbd-71a8dcc2fd7f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:09:14.302582 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:14.302498    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:14.302582 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:14.302530    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:14.302582 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:14.302540    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2nxq\" (UniqueName: \"kubernetes.io/projected/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-kube-api-access-g2nxq\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:14.302582 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:14.302549    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/727362a7-2b8f-401d-afbd-71a8dcc2fd7f-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:14.698683 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:14.698594    5820 generic.go:356] "Generic (PLEG): container finished" podID="727362a7-2b8f-401d-afbd-71a8dcc2fd7f" containerID="f7ac85c163ab31b1a5a2771728303aaa3d1ef44caf225ee09e5b187e45f08d27" exitCode=0
Oct 30 19:09:14.698683 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:14.698639    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"
Oct 30 19:09:14.698926 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:14.698681    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg" event={"ID":"727362a7-2b8f-401d-afbd-71a8dcc2fd7f","Type":"ContainerDied","Data":"f7ac85c163ab31b1a5a2771728303aaa3d1ef44caf225ee09e5b187e45f08d27"}
Oct 30 19:09:14.698926 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:14.698740    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg" event={"ID":"727362a7-2b8f-401d-afbd-71a8dcc2fd7f","Type":"ContainerDied","Data":"51df928a98832872f18039e91cfef8e217cd728d5411684694b8e38e02a72205"}
Oct 30 19:09:14.698926 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:14.698768    5820 scope.go:122] "RemoveContainer" containerID="f7ac85c163ab31b1a5a2771728303aaa3d1ef44caf225ee09e5b187e45f08d27"
Oct 30 19:09:14.710143 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:14.710124    5820 scope.go:122] "RemoveContainer" containerID="f7ac85c163ab31b1a5a2771728303aaa3d1ef44caf225ee09e5b187e45f08d27"
Oct 30 19:09:14.710425 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:09:14.710401    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7ac85c163ab31b1a5a2771728303aaa3d1ef44caf225ee09e5b187e45f08d27\": container with ID starting with f7ac85c163ab31b1a5a2771728303aaa3d1ef44caf225ee09e5b187e45f08d27 not found: ID does not exist" containerID="f7ac85c163ab31b1a5a2771728303aaa3d1ef44caf225ee09e5b187e45f08d27"
Oct 30 19:09:14.710496 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:14.710440    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7ac85c163ab31b1a5a2771728303aaa3d1ef44caf225ee09e5b187e45f08d27"} err="failed to get container status \"f7ac85c163ab31b1a5a2771728303aaa3d1ef44caf225ee09e5b187e45f08d27\": rpc error: code = NotFound desc = could not find container \"f7ac85c163ab31b1a5a2771728303aaa3d1ef44caf225ee09e5b187e45f08d27\": container with ID starting with f7ac85c163ab31b1a5a2771728303aaa3d1ef44caf225ee09e5b187e45f08d27 not found: ID does not exist"
Oct 30 19:09:14.887237 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:14.887205    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"]
Oct 30 19:09:14.936422 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:14.936388    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-842bbd-66f6f5dd59-bdctg"]
Oct 30 19:09:15.437449 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:15.437419    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="727362a7-2b8f-401d-afbd-71a8dcc2fd7f" path="/var/lib/kubelet/pods/727362a7-2b8f-401d-afbd-71a8dcc2fd7f/volumes"
Oct 30 19:09:15.600154 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:15.600114    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:15.829935 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:15.829907    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:16.004160 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:16.004129    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:16.564082 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:16.564042    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:20.776810 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:20.776719    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:23.000119 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:23.000079    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:23.176851 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:23.176809    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:23.327539 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:23.327510    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"]
Oct 30 19:09:23.327744 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:23.327722    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8" podUID="18a0d7fb-d2d5-49b6-9d2c-be64503459de" containerName="istio-proxy" containerID="cri-o://15e0c5326bfcd81f90e8fd5f05080b619e43a816fe4a1fd2df796296c6a8c60e" gracePeriod=30
Oct 30 19:09:23.362965 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:23.362907    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:25.937188 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:25.937145    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:28.590860 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.590809    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:09:28.715262 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.715217    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/18a0d7fb-d2d5-49b6-9d2c-be64503459de-kube-api-access-v74b2\" (UniqueName: \"kubernetes.io/projected/18a0d7fb-d2d5-49b6-9d2c-be64503459de-kube-api-access-v74b2\") pod \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") "
Oct 30 19:09:28.727776 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.727744    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istiod-ca-cert\") pod \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") "
Oct 30 19:09:28.728103 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.728068    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-workload-certs\") pod \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") "
Oct 30 19:09:28.728837 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.728791    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istiod-ca-cert" pod "18a0d7fb-d2d5-49b6-9d2c-be64503459de" (UID: "18a0d7fb-d2d5-49b6-9d2c-be64503459de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:09:28.736739 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.733122    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-workload-certs" pod "18a0d7fb-d2d5-49b6-9d2c-be64503459de" (UID: "18a0d7fb-d2d5-49b6-9d2c-be64503459de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:09:28.736739 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.733187    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-podinfo\") pod \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") "
Oct 30 19:09:28.736739 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.733259    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-credential-socket\") pod \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") "
Oct 30 19:09:28.736739 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.733509    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-data\") pod \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") "
Oct 30 19:09:28.736739 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.733564    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-envoy\") pod \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") "
Oct 30 19:09:28.736739 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.733616    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-workload-socket\") pod \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") "
Oct 30 19:09:28.736739 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.733652    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-token\" (UniqueName: \"kubernetes.io/projected/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-token\") pod \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\" (UID: \"18a0d7fb-d2d5-49b6-9d2c-be64503459de\") "
Oct 30 19:09:28.736739 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.734573    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:28.736739 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.734599    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:28.736739 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.734883    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-data" pod "18a0d7fb-d2d5-49b6-9d2c-be64503459de" (UID: "18a0d7fb-d2d5-49b6-9d2c-be64503459de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:09:28.737482 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.737165    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-credential-socket" pod "18a0d7fb-d2d5-49b6-9d2c-be64503459de" (UID: "18a0d7fb-d2d5-49b6-9d2c-be64503459de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:09:28.737482 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.737323    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-workload-socket" pod "18a0d7fb-d2d5-49b6-9d2c-be64503459de" (UID: "18a0d7fb-d2d5-49b6-9d2c-be64503459de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:09:28.737676 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.737653    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18a0d7fb-d2d5-49b6-9d2c-be64503459de-kube-api-access-v74b2" pod "18a0d7fb-d2d5-49b6-9d2c-be64503459de" (UID: "18a0d7fb-d2d5-49b6-9d2c-be64503459de"). InnerVolumeSpecName "kube-api-access-v74b2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:09:28.738278 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.738217    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-podinfo" pod "18a0d7fb-d2d5-49b6-9d2c-be64503459de" (UID: "18a0d7fb-d2d5-49b6-9d2c-be64503459de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:09:28.740318 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.740293    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-envoy" pod "18a0d7fb-d2d5-49b6-9d2c-be64503459de" (UID: "18a0d7fb-d2d5-49b6-9d2c-be64503459de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:09:28.740661 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.740615    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-token" pod "18a0d7fb-d2d5-49b6-9d2c-be64503459de" (UID: "18a0d7fb-d2d5-49b6-9d2c-be64503459de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:09:28.835618 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.835585    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:28.835618 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.835619    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:28.835844 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.835633    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:28.835844 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.835648    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:28.835844 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.835687    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v74b2\" (UniqueName: \"kubernetes.io/projected/18a0d7fb-d2d5-49b6-9d2c-be64503459de-kube-api-access-v74b2\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:28.835844 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.835702    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18a0d7fb-d2d5-49b6-9d2c-be64503459de-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:28.835844 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.835716    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18a0d7fb-d2d5-49b6-9d2c-be64503459de-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:28.835844 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.835799    5820 generic.go:356] "Generic (PLEG): container finished" podID="18a0d7fb-d2d5-49b6-9d2c-be64503459de" containerID="15e0c5326bfcd81f90e8fd5f05080b619e43a816fe4a1fd2df796296c6a8c60e" exitCode=0
Oct 30 19:09:28.836084 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.835865    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8" event={"ID":"18a0d7fb-d2d5-49b6-9d2c-be64503459de","Type":"ContainerDied","Data":"15e0c5326bfcd81f90e8fd5f05080b619e43a816fe4a1fd2df796296c6a8c60e"}
Oct 30 19:09:28.836084 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.835877    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"
Oct 30 19:09:28.836084 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.835973    5820 scope.go:122] "RemoveContainer" containerID="15e0c5326bfcd81f90e8fd5f05080b619e43a816fe4a1fd2df796296c6a8c60e"
Oct 30 19:09:28.836084 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.835989    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8" event={"ID":"18a0d7fb-d2d5-49b6-9d2c-be64503459de","Type":"ContainerDied","Data":"84dd47570ca8d884adfb25f29cf789827a4d53f0077793b0070646f192ec1028"}
Oct 30 19:09:28.846606 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.846585    5820 scope.go:122] "RemoveContainer" containerID="d8b91a1a2211449bc4a71de67b331434aec2c04a4e34f8a39fbb58d40438cd62"
Oct 30 19:09:28.855872 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.855854    5820 scope.go:122] "RemoveContainer" containerID="15e0c5326bfcd81f90e8fd5f05080b619e43a816fe4a1fd2df796296c6a8c60e"
Oct 30 19:09:28.856184 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:09:28.856164    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15e0c5326bfcd81f90e8fd5f05080b619e43a816fe4a1fd2df796296c6a8c60e\": container with ID starting with 15e0c5326bfcd81f90e8fd5f05080b619e43a816fe4a1fd2df796296c6a8c60e not found: ID does not exist" containerID="15e0c5326bfcd81f90e8fd5f05080b619e43a816fe4a1fd2df796296c6a8c60e"
Oct 30 19:09:28.856242 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.856193    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15e0c5326bfcd81f90e8fd5f05080b619e43a816fe4a1fd2df796296c6a8c60e"} err="failed to get container status \"15e0c5326bfcd81f90e8fd5f05080b619e43a816fe4a1fd2df796296c6a8c60e\": rpc error: code = NotFound desc = could not find container \"15e0c5326bfcd81f90e8fd5f05080b619e43a816fe4a1fd2df796296c6a8c60e\": container with ID starting with 15e0c5326bfcd81f90e8fd5f05080b619e43a816fe4a1fd2df796296c6a8c60e not found: ID does not exist"
Oct 30 19:09:28.856242 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.856211    5820 scope.go:122] "RemoveContainer" containerID="d8b91a1a2211449bc4a71de67b331434aec2c04a4e34f8a39fbb58d40438cd62"
Oct 30 19:09:28.856462 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:09:28.856438    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8b91a1a2211449bc4a71de67b331434aec2c04a4e34f8a39fbb58d40438cd62\": container with ID starting with d8b91a1a2211449bc4a71de67b331434aec2c04a4e34f8a39fbb58d40438cd62 not found: ID does not exist" containerID="d8b91a1a2211449bc4a71de67b331434aec2c04a4e34f8a39fbb58d40438cd62"
Oct 30 19:09:28.856524 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:28.856471    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8b91a1a2211449bc4a71de67b331434aec2c04a4e34f8a39fbb58d40438cd62"} err="failed to get container status \"d8b91a1a2211449bc4a71de67b331434aec2c04a4e34f8a39fbb58d40438cd62\": rpc error: code = NotFound desc = could not find container \"d8b91a1a2211449bc4a71de67b331434aec2c04a4e34f8a39fbb58d40438cd62\": container with ID starting with d8b91a1a2211449bc4a71de67b331434aec2c04a4e34f8a39fbb58d40438cd62 not found: ID does not exist"
Oct 30 19:09:29.004190 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:29.004161    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"]
Oct 30 19:09:29.009062 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:29.009028    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-842bbd-openshift-default-7f747494d5-4csw8"]
Oct 30 19:09:29.438621 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:29.438583    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="18a0d7fb-d2d5-49b6-9d2c-be64503459de" path="/var/lib/kubelet/pods/18a0d7fb-d2d5-49b6-9d2c-be64503459de/volumes"
Oct 30 19:09:32.887976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:32.886686    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:35.700017 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:35.699985    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4"]
Oct 30 19:09:35.813462 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:35.813101    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs"]
Oct 30 19:09:36.057290 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:36.057260    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:36.286118 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:36.286078    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:37.194977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.181402    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"]
Oct 30 19:09:37.194977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.181691    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7" podUID="c143e1e4-afe7-48ea-b014-d586f51bcacf" containerName="maas-api" containerID="cri-o://19327e77c5a049484e7659c9966dd7fb0f5b014a52766a543d01ebcac17ef037" gracePeriod=30
Oct 30 19:09:37.304022 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.303980    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:37.618033 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.611338    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"
Oct 30 19:09:37.642060 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.642019    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7"]
Oct 30 19:09:37.642264 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.642228    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7" podUID="19332f09-bec4-403a-9a0c-f7989f71bba7" containerName="payload-processing" containerID="cri-o://9fe6ff9c887800cf7d248efcbc9f6bea3f672fe1a0d2912f409990192c60f568" gracePeriod=30
Oct 30 19:09:37.711727 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.711689    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c143e1e4-afe7-48ea-b014-d586f51bcacf-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c143e1e4-afe7-48ea-b014-d586f51bcacf-openshift-service-ca\") pod \"c143e1e4-afe7-48ea-b014-d586f51bcacf\" (UID: \"c143e1e4-afe7-48ea-b014-d586f51bcacf\") "
Oct 30 19:09:37.711727 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.711729    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c143e1e4-afe7-48ea-b014-d586f51bcacf-kube-api-access-9x6bx\" (UniqueName: \"kubernetes.io/projected/c143e1e4-afe7-48ea-b014-d586f51bcacf-kube-api-access-9x6bx\") pod \"c143e1e4-afe7-48ea-b014-d586f51bcacf\" (UID: \"c143e1e4-afe7-48ea-b014-d586f51bcacf\") "
Oct 30 19:09:37.712039 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.711779    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c143e1e4-afe7-48ea-b014-d586f51bcacf-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c143e1e4-afe7-48ea-b014-d586f51bcacf-maas-api-tls\") pod \"c143e1e4-afe7-48ea-b014-d586f51bcacf\" (UID: \"c143e1e4-afe7-48ea-b014-d586f51bcacf\") "
Oct 30 19:09:37.712039 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.711849    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c143e1e4-afe7-48ea-b014-d586f51bcacf-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c143e1e4-afe7-48ea-b014-d586f51bcacf-metrics-certs\") pod \"c143e1e4-afe7-48ea-b014-d586f51bcacf\" (UID: \"c143e1e4-afe7-48ea-b014-d586f51bcacf\") "
Oct 30 19:09:37.712166 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.712112    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c143e1e4-afe7-48ea-b014-d586f51bcacf-openshift-service-ca" pod "c143e1e4-afe7-48ea-b014-d586f51bcacf" (UID: "c143e1e4-afe7-48ea-b014-d586f51bcacf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:09:37.712535 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.712436    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c143e1e4-afe7-48ea-b014-d586f51bcacf-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:37.714200 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.714170    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c143e1e4-afe7-48ea-b014-d586f51bcacf-maas-api-tls" pod "c143e1e4-afe7-48ea-b014-d586f51bcacf" (UID: "c143e1e4-afe7-48ea-b014-d586f51bcacf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:09:37.714334 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.714280    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c143e1e4-afe7-48ea-b014-d586f51bcacf-kube-api-access-9x6bx" pod "c143e1e4-afe7-48ea-b014-d586f51bcacf" (UID: "c143e1e4-afe7-48ea-b014-d586f51bcacf"). InnerVolumeSpecName "kube-api-access-9x6bx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:09:37.714512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.714486    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c143e1e4-afe7-48ea-b014-d586f51bcacf-metrics-certs" pod "c143e1e4-afe7-48ea-b014-d586f51bcacf" (UID: "c143e1e4-afe7-48ea-b014-d586f51bcacf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:09:37.814992 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.814597    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c143e1e4-afe7-48ea-b014-d586f51bcacf-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:37.814992 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.814635    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c143e1e4-afe7-48ea-b014-d586f51bcacf-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:37.814992 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.814651    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9x6bx\" (UniqueName: \"kubernetes.io/projected/c143e1e4-afe7-48ea-b014-d586f51bcacf-kube-api-access-9x6bx\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:37.841738 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.841697    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:37.948378 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.948340    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7"
Oct 30 19:09:37.990906 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.990866    5820 generic.go:356] "Generic (PLEG): container finished" podID="c143e1e4-afe7-48ea-b014-d586f51bcacf" containerID="19327e77c5a049484e7659c9966dd7fb0f5b014a52766a543d01ebcac17ef037" exitCode=0
Oct 30 19:09:37.991132 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.990934    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"
Oct 30 19:09:37.991132 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.990929    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7" event={"ID":"c143e1e4-afe7-48ea-b014-d586f51bcacf","Type":"ContainerDied","Data":"19327e77c5a049484e7659c9966dd7fb0f5b014a52766a543d01ebcac17ef037"}
Oct 30 19:09:37.991132 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.991072    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7" event={"ID":"c143e1e4-afe7-48ea-b014-d586f51bcacf","Type":"ContainerDied","Data":"4f7aa7e1b5bee9c34f89e5e606b8705c5d9bca602b4757851d1c2e2bcc1ce436"}
Oct 30 19:09:37.991132 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.991105    5820 scope.go:122] "RemoveContainer" containerID="19327e77c5a049484e7659c9966dd7fb0f5b014a52766a543d01ebcac17ef037"
Oct 30 19:09:37.992608 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.992584    5820 generic.go:356] "Generic (PLEG): container finished" podID="19332f09-bec4-403a-9a0c-f7989f71bba7" containerID="9fe6ff9c887800cf7d248efcbc9f6bea3f672fe1a0d2912f409990192c60f568" exitCode=0
Oct 30 19:09:37.992733 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.992659    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7"
Oct 30 19:09:37.992733 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.992671    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7" event={"ID":"19332f09-bec4-403a-9a0c-f7989f71bba7","Type":"ContainerDied","Data":"9fe6ff9c887800cf7d248efcbc9f6bea3f672fe1a0d2912f409990192c60f568"}
Oct 30 19:09:37.992733 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:37.992720    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7" event={"ID":"19332f09-bec4-403a-9a0c-f7989f71bba7","Type":"ContainerDied","Data":"6dad8a3ef1e5b7b039a42ec4b590819e30dd778ca77a828766b5107a1460291d"}
Oct 30 19:09:38.003576 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:38.003553    5820 scope.go:122] "RemoveContainer" containerID="19327e77c5a049484e7659c9966dd7fb0f5b014a52766a543d01ebcac17ef037"
Oct 30 19:09:38.003882 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:09:38.003860    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19327e77c5a049484e7659c9966dd7fb0f5b014a52766a543d01ebcac17ef037\": container with ID starting with 19327e77c5a049484e7659c9966dd7fb0f5b014a52766a543d01ebcac17ef037 not found: ID does not exist" containerID="19327e77c5a049484e7659c9966dd7fb0f5b014a52766a543d01ebcac17ef037"
Oct 30 19:09:38.003979 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:38.003889    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19327e77c5a049484e7659c9966dd7fb0f5b014a52766a543d01ebcac17ef037"} err="failed to get container status \"19327e77c5a049484e7659c9966dd7fb0f5b014a52766a543d01ebcac17ef037\": rpc error: code = NotFound desc = could not find container \"19327e77c5a049484e7659c9966dd7fb0f5b014a52766a543d01ebcac17ef037\": container with ID starting with 19327e77c5a049484e7659c9966dd7fb0f5b014a52766a543d01ebcac17ef037 not found: ID does not exist"
Oct 30 19:09:38.003979 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:38.003905    5820 scope.go:122] "RemoveContainer" containerID="9fe6ff9c887800cf7d248efcbc9f6bea3f672fe1a0d2912f409990192c60f568"
Oct 30 19:09:38.015287 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:38.015268    5820 scope.go:122] "RemoveContainer" containerID="9fe6ff9c887800cf7d248efcbc9f6bea3f672fe1a0d2912f409990192c60f568"
Oct 30 19:09:38.015575 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:09:38.015552    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fe6ff9c887800cf7d248efcbc9f6bea3f672fe1a0d2912f409990192c60f568\": container with ID starting with 9fe6ff9c887800cf7d248efcbc9f6bea3f672fe1a0d2912f409990192c60f568 not found: ID does not exist" containerID="9fe6ff9c887800cf7d248efcbc9f6bea3f672fe1a0d2912f409990192c60f568"
Oct 30 19:09:38.015676 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:38.015579    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fe6ff9c887800cf7d248efcbc9f6bea3f672fe1a0d2912f409990192c60f568"} err="failed to get container status \"9fe6ff9c887800cf7d248efcbc9f6bea3f672fe1a0d2912f409990192c60f568\": rpc error: code = NotFound desc = could not find container \"9fe6ff9c887800cf7d248efcbc9f6bea3f672fe1a0d2912f409990192c60f568\": container with ID starting with 9fe6ff9c887800cf7d248efcbc9f6bea3f672fe1a0d2912f409990192c60f568 not found: ID does not exist"
Oct 30 19:09:38.016920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:38.016901    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19332f09-bec4-403a-9a0c-f7989f71bba7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19332f09-bec4-403a-9a0c-f7989f71bba7-plugins-config-volume\") pod \"19332f09-bec4-403a-9a0c-f7989f71bba7\" (UID: \"19332f09-bec4-403a-9a0c-f7989f71bba7\") "
Oct 30 19:09:38.017097 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:38.017081    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19332f09-bec4-403a-9a0c-f7989f71bba7-kube-api-access-5gzbp\" (UniqueName: \"kubernetes.io/projected/19332f09-bec4-403a-9a0c-f7989f71bba7-kube-api-access-5gzbp\") pod \"19332f09-bec4-403a-9a0c-f7989f71bba7\" (UID: \"19332f09-bec4-403a-9a0c-f7989f71bba7\") "
Oct 30 19:09:38.017264 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:38.017246    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19332f09-bec4-403a-9a0c-f7989f71bba7-plugins-config-volume" pod "19332f09-bec4-403a-9a0c-f7989f71bba7" (UID: "19332f09-bec4-403a-9a0c-f7989f71bba7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:09:38.017493 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:38.017478    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19332f09-bec4-403a-9a0c-f7989f71bba7-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:38.019280 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:38.019260    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19332f09-bec4-403a-9a0c-f7989f71bba7-kube-api-access-5gzbp" pod "19332f09-bec4-403a-9a0c-f7989f71bba7" (UID: "19332f09-bec4-403a-9a0c-f7989f71bba7"). InnerVolumeSpecName "kube-api-access-5gzbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:09:38.021181 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:38.021154    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:38.118849 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:38.118763    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gzbp\" (UniqueName: \"kubernetes.io/projected/19332f09-bec4-403a-9a0c-f7989f71bba7-kube-api-access-5gzbp\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:38.254645 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:38.248428    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"]
Oct 30 19:09:38.261467 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:38.261427    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0ee379-775fbc5f9b-79kw7"]
Oct 30 19:09:38.288041 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:09:38.288007    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:09:38.416575 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:38.416499    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7"]
Oct 30 19:09:38.424529 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:38.424501    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0ee379-67cd45c4cd-qwqn7"]
Oct 30 19:09:38.954806 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:38.954775    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:39.257493 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:39.257397    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:39.437826 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:39.437792    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="19332f09-bec4-403a-9a0c-f7989f71bba7" path="/var/lib/kubelet/pods/19332f09-bec4-403a-9a0c-f7989f71bba7/volumes"
Oct 30 19:09:39.438328 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:39.438306    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c143e1e4-afe7-48ea-b014-d586f51bcacf" path="/var/lib/kubelet/pods/c143e1e4-afe7-48ea-b014-d586f51bcacf/volumes"
Oct 30 19:09:39.453621 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:39.453593    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:39.910123 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:39.910091    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:40.700753 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:40.700713    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4" podUID="5136fb9d-676a-47b3-8c02-a48ea1ffba82" containerName="payload-processing" containerID="cri-o://6c4fdebd47c460febf19fb46dbe9e1e750686939964d74abdd9137bf1da0ac0c" gracePeriod=25
Oct 30 19:09:40.814455 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:40.814423    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs" podUID="6a9b9d94-81c3-42d9-a52d-269ed196dd90" containerName="payload-pre-processing" containerID="cri-o://64a93225c75600500a55eefbffd2340af0546daa9f0323a705d1d2e7f1c4b78f" gracePeriod=25
Oct 30 19:09:40.962158 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:40.962093    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4"
Oct 30 19:09:41.024315 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.024279    5820 generic.go:356] "Generic (PLEG): container finished" podID="99f1797d-6313-4c12-a997-2c70b1c70d92" containerID="8573b112d9779ec6306a4dcc10578c21f9b0f2a0c7f3bff6aff99a758b1f5d22" exitCode=0
Oct 30 19:09:41.024497 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.024352    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb" event={"ID":"99f1797d-6313-4c12-a997-2c70b1c70d92","Type":"ContainerDied","Data":"8573b112d9779ec6306a4dcc10578c21f9b0f2a0c7f3bff6aff99a758b1f5d22"}
Oct 30 19:09:41.024919 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.024895    5820 scope.go:122] "RemoveContainer" containerID="8573b112d9779ec6306a4dcc10578c21f9b0f2a0c7f3bff6aff99a758b1f5d22"
Oct 30 19:09:41.027030 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.026995    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:09:41.027131 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.027085    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:09:41.027186 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.027137    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:09:41.028358 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.028338    5820 generic.go:356] "Generic (PLEG): container finished" podID="5136fb9d-676a-47b3-8c02-a48ea1ffba82" containerID="6c4fdebd47c460febf19fb46dbe9e1e750686939964d74abdd9137bf1da0ac0c" exitCode=0
Oct 30 19:09:41.028498 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.028483    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4"
Oct 30 19:09:41.028650 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.028634    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4" event={"ID":"5136fb9d-676a-47b3-8c02-a48ea1ffba82","Type":"ContainerDied","Data":"6c4fdebd47c460febf19fb46dbe9e1e750686939964d74abdd9137bf1da0ac0c"}
Oct 30 19:09:41.028722 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.028677    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4" event={"ID":"5136fb9d-676a-47b3-8c02-a48ea1ffba82","Type":"ContainerDied","Data":"9030c7054201d87a30bfddbc014e90dd6f4370075ae3696b8fedd111e68b4e46"}
Oct 30 19:09:41.028722 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.028709    5820 scope.go:122] "RemoveContainer" containerID="6c4fdebd47c460febf19fb46dbe9e1e750686939964d74abdd9137bf1da0ac0c"
Oct 30 19:09:41.032054 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.032032    5820 generic.go:356] "Generic (PLEG): container finished" podID="6a9b9d94-81c3-42d9-a52d-269ed196dd90" containerID="64a93225c75600500a55eefbffd2340af0546daa9f0323a705d1d2e7f1c4b78f" exitCode=0
Oct 30 19:09:41.032145 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.032083    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs" event={"ID":"6a9b9d94-81c3-42d9-a52d-269ed196dd90","Type":"ContainerDied","Data":"64a93225c75600500a55eefbffd2340af0546daa9f0323a705d1d2e7f1c4b78f"}
Oct 30 19:09:41.043398 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.043373    5820 scope.go:122] "RemoveContainer" containerID="6c4fdebd47c460febf19fb46dbe9e1e750686939964d74abdd9137bf1da0ac0c"
Oct 30 19:09:41.043699 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:09:41.043666    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c4fdebd47c460febf19fb46dbe9e1e750686939964d74abdd9137bf1da0ac0c\": container with ID starting with 6c4fdebd47c460febf19fb46dbe9e1e750686939964d74abdd9137bf1da0ac0c not found: ID does not exist" containerID="6c4fdebd47c460febf19fb46dbe9e1e750686939964d74abdd9137bf1da0ac0c"
Oct 30 19:09:41.043799 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.043713    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c4fdebd47c460febf19fb46dbe9e1e750686939964d74abdd9137bf1da0ac0c"} err="failed to get container status \"6c4fdebd47c460febf19fb46dbe9e1e750686939964d74abdd9137bf1da0ac0c\": rpc error: code = NotFound desc = could not find container \"6c4fdebd47c460febf19fb46dbe9e1e750686939964d74abdd9137bf1da0ac0c\": container with ID starting with 6c4fdebd47c460febf19fb46dbe9e1e750686939964d74abdd9137bf1da0ac0c not found: ID does not exist"
Oct 30 19:09:41.092995 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.092917    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5136fb9d-676a-47b3-8c02-a48ea1ffba82-kube-api-access-lnhcq\" (UniqueName: \"kubernetes.io/projected/5136fb9d-676a-47b3-8c02-a48ea1ffba82-kube-api-access-lnhcq\") pod \"5136fb9d-676a-47b3-8c02-a48ea1ffba82\" (UID: \"5136fb9d-676a-47b3-8c02-a48ea1ffba82\") "
Oct 30 19:09:41.093131 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.093029    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5136fb9d-676a-47b3-8c02-a48ea1ffba82-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5136fb9d-676a-47b3-8c02-a48ea1ffba82-plugins-config-volume\") pod \"5136fb9d-676a-47b3-8c02-a48ea1ffba82\" (UID: \"5136fb9d-676a-47b3-8c02-a48ea1ffba82\") "
Oct 30 19:09:41.093902 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.093874    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5136fb9d-676a-47b3-8c02-a48ea1ffba82-plugins-config-volume" pod "5136fb9d-676a-47b3-8c02-a48ea1ffba82" (UID: "5136fb9d-676a-47b3-8c02-a48ea1ffba82"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:09:41.095934 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.095906    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5136fb9d-676a-47b3-8c02-a48ea1ffba82-kube-api-access-lnhcq" pod "5136fb9d-676a-47b3-8c02-a48ea1ffba82" (UID: "5136fb9d-676a-47b3-8c02-a48ea1ffba82"). InnerVolumeSpecName "kube-api-access-lnhcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:09:41.121992 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.121968    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs"
Oct 30 19:09:41.194373 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.194341    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a9b9d94-81c3-42d9-a52d-269ed196dd90-kube-api-access-vn48b\" (UniqueName: \"kubernetes.io/projected/6a9b9d94-81c3-42d9-a52d-269ed196dd90-kube-api-access-vn48b\") pod \"6a9b9d94-81c3-42d9-a52d-269ed196dd90\" (UID: \"6a9b9d94-81c3-42d9-a52d-269ed196dd90\") "
Oct 30 19:09:41.194557 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.194420    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a9b9d94-81c3-42d9-a52d-269ed196dd90-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a9b9d94-81c3-42d9-a52d-269ed196dd90-plugins-config-volume\") pod \"6a9b9d94-81c3-42d9-a52d-269ed196dd90\" (UID: \"6a9b9d94-81c3-42d9-a52d-269ed196dd90\") "
Oct 30 19:09:41.194863 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.194827    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a9b9d94-81c3-42d9-a52d-269ed196dd90-plugins-config-volume" pod "6a9b9d94-81c3-42d9-a52d-269ed196dd90" (UID: "6a9b9d94-81c3-42d9-a52d-269ed196dd90"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:09:41.195029 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.195008    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnhcq\" (UniqueName: \"kubernetes.io/projected/5136fb9d-676a-47b3-8c02-a48ea1ffba82-kube-api-access-lnhcq\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:41.195093 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.195035    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5136fb9d-676a-47b3-8c02-a48ea1ffba82-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:41.195093 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.195050    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a9b9d94-81c3-42d9-a52d-269ed196dd90-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:41.196637 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.196613    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a9b9d94-81c3-42d9-a52d-269ed196dd90-kube-api-access-vn48b" pod "6a9b9d94-81c3-42d9-a52d-269ed196dd90" (UID: "6a9b9d94-81c3-42d9-a52d-269ed196dd90"). InnerVolumeSpecName "kube-api-access-vn48b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:09:41.297179 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.297101    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vn48b\" (UniqueName: \"kubernetes.io/projected/6a9b9d94-81c3-42d9-a52d-269ed196dd90-kube-api-access-vn48b\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:41.482646 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.482616    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4"]
Oct 30 19:09:41.493655 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:41.493624    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e67448-7d8b6f987b-bknm4"]
Oct 30 19:09:42.064841 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:42.064808    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb" event={"ID":"99f1797d-6313-4c12-a997-2c70b1c70d92","Type":"ContainerStarted","Data":"41e6824f7c468347dc036a30bc866b6609deefdb899a236c50da5ab28abfa4e6"}
Oct 30 19:09:42.067429 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:42.067405    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs" event={"ID":"6a9b9d94-81c3-42d9-a52d-269ed196dd90","Type":"ContainerDied","Data":"10dcd6615a0b0925cf0dbc9a7deed1fbe8fbfc6c528dcd737904876845adefee"}
Oct 30 19:09:42.067574 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:42.067431    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs"
Oct 30 19:09:42.067574 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:42.067456    5820 scope.go:122] "RemoveContainer" containerID="64a93225c75600500a55eefbffd2340af0546daa9f0323a705d1d2e7f1c4b78f"
Oct 30 19:09:42.311758 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:42.311722    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs"]
Oct 30 19:09:42.316582 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:42.316519    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e67448-6bd6c5bb4c-4znbs"]
Oct 30 19:09:42.963897 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:42.963855    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:09:42.964153 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:42.964074    5820 patch_prober.go:29] interesting pod/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Oct 30 19:09:42.964153 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:42.964126    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb" podUID="99f1797d-6313-4c12-a997-2c70b1c70d92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Oct 30 19:09:43.459608 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:43.459572    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5136fb9d-676a-47b3-8c02-a48ea1ffba82" path="/var/lib/kubelet/pods/5136fb9d-676a-47b3-8c02-a48ea1ffba82/volumes"
Oct 30 19:09:43.460171 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:43.460075    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a9b9d94-81c3-42d9-a52d-269ed196dd90" path="/var/lib/kubelet/pods/6a9b9d94-81c3-42d9-a52d-269ed196dd90/volumes"
Oct 30 19:09:43.963433 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:43.963388    5820 patch_prober.go:29] interesting pod/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Oct 30 19:09:43.963686 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:43.963446    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb" podUID="99f1797d-6313-4c12-a997-2c70b1c70d92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Oct 30 19:09:44.963607 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:44.963563    5820 patch_prober.go:29] interesting pod/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Oct 30 19:09:44.964255 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:44.963637    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb" podUID="99f1797d-6313-4c12-a997-2c70b1c70d92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Oct 30 19:09:45.963401 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:45.963359    5820 patch_prober.go:29] interesting pod/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Oct 30 19:09:45.963617 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:45.963432    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb" podUID="99f1797d-6313-4c12-a997-2c70b1c70d92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Oct 30 19:09:46.516575 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:46.516536    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:46.697160 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:46.697117    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:46.885937 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:46.885897    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:46.974392 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:46.974348    5820 patch_prober.go:29] interesting pod/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Oct 30 19:09:46.975582 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:46.975546    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb" podUID="99f1797d-6313-4c12-a997-2c70b1c70d92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Oct 30 19:09:47.967709 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:47.967680    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:09:47.968107 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:47.968059    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:09:47.968814 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:47.968798    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:09:48.376442 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:48.376383    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:49.413363 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.413329    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"]
Oct 30 19:09:49.413928 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.413915    5820 cpu_manager.go:414] "containerMap: removing container" podUID="5136fb9d-676a-47b3-8c02-a48ea1ffba82" containerName="payload-processing"
Oct 30 19:09:49.414014 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.413929    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5136fb9d-676a-47b3-8c02-a48ea1ffba82" containerName="payload-processing"
Oct 30 19:09:49.414014 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.413958    5820 cpu_manager.go:414] "containerMap: removing container" podUID="18a0d7fb-d2d5-49b6-9d2c-be64503459de" containerName="istio-proxy"
Oct 30 19:09:49.414014 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.413966    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="18a0d7fb-d2d5-49b6-9d2c-be64503459de" containerName="istio-proxy"
Oct 30 19:09:49.414014 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.413980    5820 cpu_manager.go:414] "containerMap: removing container" podUID="19332f09-bec4-403a-9a0c-f7989f71bba7" containerName="payload-processing"
Oct 30 19:09:49.414014 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.413987    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19332f09-bec4-403a-9a0c-f7989f71bba7" containerName="payload-processing"
Oct 30 19:09:49.414172 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.414019    5820 cpu_manager.go:414] "containerMap: removing container" podUID="727362a7-2b8f-401d-afbd-71a8dcc2fd7f" containerName="maas-api"
Oct 30 19:09:49.414172 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.414024    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="727362a7-2b8f-401d-afbd-71a8dcc2fd7f" containerName="maas-api"
Oct 30 19:09:49.414172 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.414054    5820 cpu_manager.go:414] "containerMap: removing container" podUID="c143e1e4-afe7-48ea-b014-d586f51bcacf" containerName="maas-api"
Oct 30 19:09:49.414172 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.414058    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c143e1e4-afe7-48ea-b014-d586f51bcacf" containerName="maas-api"
Oct 30 19:09:49.414172 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.414071    5820 cpu_manager.go:414] "containerMap: removing container" podUID="6a9b9d94-81c3-42d9-a52d-269ed196dd90" containerName="payload-pre-processing"
Oct 30 19:09:49.414172 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.414077    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a9b9d94-81c3-42d9-a52d-269ed196dd90" containerName="payload-pre-processing"
Oct 30 19:09:49.414172 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.414167    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="727362a7-2b8f-401d-afbd-71a8dcc2fd7f" containerName="maas-api"
Oct 30 19:09:49.414391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.414180    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="c143e1e4-afe7-48ea-b014-d586f51bcacf" containerName="maas-api"
Oct 30 19:09:49.414391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.414188    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="18a0d7fb-d2d5-49b6-9d2c-be64503459de" containerName="istio-proxy"
Oct 30 19:09:49.414391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.414194    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a9b9d94-81c3-42d9-a52d-269ed196dd90" containerName="payload-pre-processing"
Oct 30 19:09:49.414391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.414201    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="5136fb9d-676a-47b3-8c02-a48ea1ffba82" containerName="payload-processing"
Oct 30 19:09:49.414391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.414207    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="18a0d7fb-d2d5-49b6-9d2c-be64503459de" containerName="istio-proxy"
Oct 30 19:09:49.414391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.414213    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="19332f09-bec4-403a-9a0c-f7989f71bba7" containerName="payload-processing"
Oct 30 19:09:49.419009 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.418991    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.471217 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.471179    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-0be4a4-openshift-default-dockercfg-q4rmn\""
Oct 30 19:09:49.526939 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.526902    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"]
Oct 30 19:09:49.538245 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.538211    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-credential-socket\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.538609 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.538557    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-workload-socket\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.538926 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.538905    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-envoy\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.539129 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.539111    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-workload-certs\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.539242 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.539228    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-token\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.539356 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.539344    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-podinfo\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.539462 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.539450    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istiod-ca-cert\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.540203 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.539933    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsrxj\" (UniqueName: \"kubernetes.io/projected/4a0996ab-5aaa-4b71-b064-87d9d1572d13-kube-api-access-vsrxj\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.540203 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.540156    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-data\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.541432 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.541391    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:49.666647 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.666519    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-data\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.671161 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.667629    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-credential-socket\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.671161 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.667703    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-workload-socket\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.671161 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.667766    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-envoy\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.671161 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.667803    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-workload-certs\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.671161 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.667823    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-token\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.671161 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.667847    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-podinfo\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.671161 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.667882    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istiod-ca-cert\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.671161 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.667940    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsrxj\" (UniqueName: \"kubernetes.io/projected/4a0996ab-5aaa-4b71-b064-87d9d1572d13-kube-api-access-vsrxj\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.675997 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.675963    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-workload-certs\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.678910 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.678873    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istiod-ca-cert\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.680850 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.679453    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-credential-socket\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.680850 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.679727    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-data\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.680850 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.679768    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-workload-socket\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.683085 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.683064    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-envoy\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.683689 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.683669    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-podinfo\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.721187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.721156    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:49.736234 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.736208    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-token\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:49.740753 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:49.740722    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsrxj\" (UniqueName: \"kubernetes.io/projected/4a0996ab-5aaa-4b71-b064-87d9d1572d13-kube-api-access-vsrxj\") pod \"e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") " pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:50.033473 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:50.033401    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:50.472218 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:50.472191    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"]
Oct 30 19:09:50.475576 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:09:50.475546    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a0996ab_5aaa_4b71_b064_87d9d1572d13.slice/crio-f08f3689ecb2162ad41fd9d2d8751faa0ac34544fab1a65f63b5aad04569285f WatchSource:0}: Error finding container f08f3689ecb2162ad41fd9d2d8751faa0ac34544fab1a65f63b5aad04569285f: Status 404 returned error can't find the container with id f08f3689ecb2162ad41fd9d2d8751faa0ac34544fab1a65f63b5aad04569285f
Oct 30 19:09:50.478662 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:50.478626    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:09:50.478764 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:50.478697    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:09:50.478764 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:50.478727    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:09:51.131358 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.131322    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6" event={"ID":"4a0996ab-5aaa-4b71-b064-87d9d1572d13","Type":"ContainerStarted","Data":"fa6c9f30b0bb40402b99756c717058cc797c78f47ad97cb081f4876bd879ea3c"}
Oct 30 19:09:51.131530 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.131376    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6" event={"ID":"4a0996ab-5aaa-4b71-b064-87d9d1572d13","Type":"ContainerStarted","Data":"f08f3689ecb2162ad41fd9d2d8751faa0ac34544fab1a65f63b5aad04569285f"}
Oct 30 19:09:51.167660 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.167629    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"]
Oct 30 19:09:51.168258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.168243    5820 cpu_manager.go:414] "containerMap: removing container" podUID="18a0d7fb-d2d5-49b6-9d2c-be64503459de" containerName="istio-proxy"
Oct 30 19:09:51.168315 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.168260    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="18a0d7fb-d2d5-49b6-9d2c-be64503459de" containerName="istio-proxy"
Oct 30 19:09:51.171843 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.171825    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"
Oct 30 19:09:51.209064 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.209038    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d71958e-32bc-43db-af6d-157ebd0af8eb-metrics-certs\") pod \"maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4\" (UID: \"6d71958e-32bc-43db-af6d-157ebd0af8eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"
Oct 30 19:09:51.209212 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.209083    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d71958e-32bc-43db-af6d-157ebd0af8eb-maas-api-tls\") pod \"maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4\" (UID: \"6d71958e-32bc-43db-af6d-157ebd0af8eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"
Oct 30 19:09:51.209361 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.209343    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d71958e-32bc-43db-af6d-157ebd0af8eb-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4\" (UID: \"6d71958e-32bc-43db-af6d-157ebd0af8eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"
Oct 30 19:09:51.209444 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.209423    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74rsr\" (UniqueName: \"kubernetes.io/projected/6d71958e-32bc-43db-af6d-157ebd0af8eb-kube-api-access-74rsr\") pod \"maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4\" (UID: \"6d71958e-32bc-43db-af6d-157ebd0af8eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"
Oct 30 19:09:51.227269 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.227246    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-7f44c7-serving-cert\""
Oct 30 19:09:51.231272 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.231248    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"]
Oct 30 19:09:51.286650 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.286583    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6" podStartSLOduration=2.286563168 podStartE2EDuration="2.286563168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:09:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:09:51.281661247 +0000 UTC m=+2111.993409947" watchObservedRunningTime="2026-10-30 19:09:51.286563168 +0000 UTC m=+2111.998311869"
Oct 30 19:09:51.310695 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.310666    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d71958e-32bc-43db-af6d-157ebd0af8eb-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4\" (UID: \"6d71958e-32bc-43db-af6d-157ebd0af8eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"
Oct 30 19:09:51.310847 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.310719    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74rsr\" (UniqueName: \"kubernetes.io/projected/6d71958e-32bc-43db-af6d-157ebd0af8eb-kube-api-access-74rsr\") pod \"maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4\" (UID: \"6d71958e-32bc-43db-af6d-157ebd0af8eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"
Oct 30 19:09:51.310847 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.310763    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d71958e-32bc-43db-af6d-157ebd0af8eb-metrics-certs\") pod \"maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4\" (UID: \"6d71958e-32bc-43db-af6d-157ebd0af8eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"
Oct 30 19:09:51.310847 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.310785    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d71958e-32bc-43db-af6d-157ebd0af8eb-maas-api-tls\") pod \"maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4\" (UID: \"6d71958e-32bc-43db-af6d-157ebd0af8eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"
Oct 30 19:09:51.311027 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:09:51.311008    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-serving-cert: secret "maas-api-e2e-ipp-b-7f44c7-serving-cert" not found
Oct 30 19:09:51.311081 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:09:51.311068    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d71958e-32bc-43db-af6d-157ebd0af8eb-maas-api-tls podName:6d71958e-32bc-43db-af6d-157ebd0af8eb nodeName:}" failed. No retries permitted until 2026-10-30 19:09:51.811053945 +0000 UTC m=+2112.522802633 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6d71958e-32bc-43db-af6d-157ebd0af8eb-maas-api-tls") pod "maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4" (UID: "6d71958e-32bc-43db-af6d-157ebd0af8eb") : secret "maas-api-e2e-ipp-b-7f44c7-serving-cert" not found
Oct 30 19:09:51.311510 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.311448    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d71958e-32bc-43db-af6d-157ebd0af8eb-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4\" (UID: \"6d71958e-32bc-43db-af6d-157ebd0af8eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"
Oct 30 19:09:51.313359 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.313340    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d71958e-32bc-43db-af6d-157ebd0af8eb-metrics-certs\") pod \"maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4\" (UID: \"6d71958e-32bc-43db-af6d-157ebd0af8eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"
Oct 30 19:09:51.366508 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.366473    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:51.372915 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.372880    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74rsr\" (UniqueName: \"kubernetes.io/projected/6d71958e-32bc-43db-af6d-157ebd0af8eb-kube-api-access-74rsr\") pod \"maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4\" (UID: \"6d71958e-32bc-43db-af6d-157ebd0af8eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"
Oct 30 19:09:51.598844 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.598746    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:51.772575 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.772537    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:51.824362 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.824334    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d71958e-32bc-43db-af6d-157ebd0af8eb-maas-api-tls\") pod \"maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4\" (UID: \"6d71958e-32bc-43db-af6d-157ebd0af8eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"
Oct 30 19:09:51.826997 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:51.826972    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d71958e-32bc-43db-af6d-157ebd0af8eb-maas-api-tls\") pod \"maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4\" (UID: \"6d71958e-32bc-43db-af6d-157ebd0af8eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"
Oct 30 19:09:52.033944 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:52.033871    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:52.059316 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:52.059286    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:52.086576 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:52.086541    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"
Oct 30 19:09:52.168260 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:52.166054    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:52.168260 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:52.166997    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:09:52.508891 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:09:52.508856    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d71958e_32bc_43db_af6d_157ebd0af8eb.slice/crio-6fb59fdee89a0642c0b5e6e4c8e227bec7dbc8dfd1701b57d62bbe11f9f21d37 WatchSource:0}: Error finding container 6fb59fdee89a0642c0b5e6e4c8e227bec7dbc8dfd1701b57d62bbe11f9f21d37: Status 404 returned error can't find the container with id 6fb59fdee89a0642c0b5e6e4c8e227bec7dbc8dfd1701b57d62bbe11f9f21d37
Oct 30 19:09:52.517504 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:52.517480    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"]
Oct 30 19:09:53.173444 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:53.173406    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4" event={"ID":"6d71958e-32bc-43db-af6d-157ebd0af8eb","Type":"ContainerStarted","Data":"6fb59fdee89a0642c0b5e6e4c8e227bec7dbc8dfd1701b57d62bbe11f9f21d37"}
Oct 30 19:09:54.181172 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:54.181138    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4" event={"ID":"6d71958e-32bc-43db-af6d-157ebd0af8eb","Type":"ContainerStarted","Data":"83fd5e125a51561273ef4d6289392f1389ee8c3bf747b3e4bcc89ba04cea823f"}
Oct 30 19:09:54.181609 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:54.181491    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"
Oct 30 19:09:54.283702 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:54.283658    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:54.300408 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:54.300347    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4" podStartSLOduration=2.405364297 podStartE2EDuration="3.300332849s" totalImagesPullingTime="894.968552ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:09:54.298171891 +0000 UTC m=+2115.009920588" watchObservedRunningTime="2026-10-30 19:09:54.300332849 +0000 UTC m=+2115.012081547"
Oct 30 19:09:54.466095 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:54.465397    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:54.873699 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:54.873661    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:56.058977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:56.046591    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:56.953334 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:56.953298    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:57.178846 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:57.178791    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:58.327277 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:58.327243    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:59.737273 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:59.737234    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:59.931006 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:09:59.930965    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:00.118564 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:00.118524    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr"]
Oct 30 19:10:00.123919 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:00.123887    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr"
Oct 30 19:10:00.178589 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:00.178550    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr"]
Oct 30 19:10:00.194859 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:00.194827    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"
Oct 30 19:10:00.216917 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:00.216873    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g6rr\" (UniqueName: \"kubernetes.io/projected/5b9f46cd-813e-40f5-b4ec-6bea1b5bcead-kube-api-access-9g6rr\") pod \"payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr\" (UID: \"5b9f46cd-813e-40f5-b4ec-6bea1b5bcead\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr"
Oct 30 19:10:00.217250 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:00.217227    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b9f46cd-813e-40f5-b4ec-6bea1b5bcead-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr\" (UID: \"5b9f46cd-813e-40f5-b4ec-6bea1b5bcead\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr"
Oct 30 19:10:00.224582 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:00.224555    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-a-0be4a4\""
Oct 30 19:10:00.319048 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:00.319010    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b9f46cd-813e-40f5-b4ec-6bea1b5bcead-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr\" (UID: \"5b9f46cd-813e-40f5-b4ec-6bea1b5bcead\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr"
Oct 30 19:10:00.320080 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:00.319115    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g6rr\" (UniqueName: \"kubernetes.io/projected/5b9f46cd-813e-40f5-b4ec-6bea1b5bcead-kube-api-access-9g6rr\") pod \"payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr\" (UID: \"5b9f46cd-813e-40f5-b4ec-6bea1b5bcead\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr"
Oct 30 19:10:00.327868 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:00.320488    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b9f46cd-813e-40f5-b4ec-6bea1b5bcead-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr\" (UID: \"5b9f46cd-813e-40f5-b4ec-6bea1b5bcead\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr"
Oct 30 19:10:00.384850 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:00.381604    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g6rr\" (UniqueName: \"kubernetes.io/projected/5b9f46cd-813e-40f5-b4ec-6bea1b5bcead-kube-api-access-9g6rr\") pod \"payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr\" (UID: \"5b9f46cd-813e-40f5-b4ec-6bea1b5bcead\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr"
Oct 30 19:10:00.439752 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:00.439723    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr"
Oct 30 19:10:00.833504 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:00.833480    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr"]
Oct 30 19:10:00.835434 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:10:00.835400    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b9f46cd_813e_40f5_b4ec_6bea1b5bcead.slice/crio-a09f3c9268125ba0103eb87dd89a034d91552f4c54fe18b44accd6541cbf88c8 WatchSource:0}: Error finding container a09f3c9268125ba0103eb87dd89a034d91552f4c54fe18b44accd6541cbf88c8: Status 404 returned error can't find the container with id a09f3c9268125ba0103eb87dd89a034d91552f4c54fe18b44accd6541cbf88c8
Oct 30 19:10:01.231747 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:01.231647    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr" event={"ID":"5b9f46cd-813e-40f5-b4ec-6bea1b5bcead","Type":"ContainerStarted","Data":"54e1855315f3baae3482fc89118d1d46e0029841ca5887cca4a5c3f1b4456c9f"}
Oct 30 19:10:01.231747 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:01.231713    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr"
Oct 30 19:10:01.231989 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:01.231753    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr" event={"ID":"5b9f46cd-813e-40f5-b4ec-6bea1b5bcead","Type":"ContainerStarted","Data":"a09f3c9268125ba0103eb87dd89a034d91552f4c54fe18b44accd6541cbf88c8"}
Oct 30 19:10:01.233404 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:01.233374    5820 generic.go:356] "Generic (PLEG): container finished" podID="9384e565-f55a-45e3-bd08-b86c27972b91" containerID="2573b5fca0645a9b31cdfd7306fb8d45beb95427a44e566994424f4e784547f9" exitCode=0
Oct 30 19:10:01.233502 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:01.233443    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch" event={"ID":"9384e565-f55a-45e3-bd08-b86c27972b91","Type":"ContainerDied","Data":"2573b5fca0645a9b31cdfd7306fb8d45beb95427a44e566994424f4e784547f9"}
Oct 30 19:10:01.233780 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:01.233766    5820 scope.go:122] "RemoveContainer" containerID="2573b5fca0645a9b31cdfd7306fb8d45beb95427a44e566994424f4e784547f9"
Oct 30 19:10:01.235178 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:01.235152    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:10:01.235246 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:01.235234    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:10:01.235284 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:01.235267    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:10:01.339785 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:01.339750    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:01.348911 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:01.348623    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr" podStartSLOduration=1.348607189 podStartE2EDuration="1.348607189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:10:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:10:01.347396073 +0000 UTC m=+2122.059144771" watchObservedRunningTime="2026-10-30 19:10:01.348607189 +0000 UTC m=+2122.060355888"
Oct 30 19:10:01.532568 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:01.532477    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:02.127021 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:02.126910    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:02.266679 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:02.266635    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch" event={"ID":"9384e565-f55a-45e3-bd08-b86c27972b91","Type":"ContainerStarted","Data":"d42a61d9faf94cff5544b1f828baf70555b702c7c9e2b6d9d131044185cec452"}
Oct 30 19:10:02.929885 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:02.929848    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:10:02.930136 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:02.930096    5820 patch_prober.go:29] interesting pod/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 30 19:10:02.930215 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:02.930168    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch" podUID="9384e565-f55a-45e3-bd08-b86c27972b91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 30 19:10:03.929774 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:03.929740    5820 patch_prober.go:29] interesting pod/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 30 19:10:03.930256 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:03.929805    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch" podUID="9384e565-f55a-45e3-bd08-b86c27972b91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 30 19:10:04.929591 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:04.929559    5820 patch_prober.go:29] interesting pod/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 30 19:10:04.929782 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:04.929612    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch" podUID="9384e565-f55a-45e3-bd08-b86c27972b91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 30 19:10:05.929930 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:05.929898    5820 patch_prober.go:29] interesting pod/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 30 19:10:05.930334 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:05.929968    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch" podUID="9384e565-f55a-45e3-bd08-b86c27972b91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 30 19:10:06.930372 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:06.930339    5820 patch_prober.go:29] interesting pod/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 30 19:10:06.930772 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:06.930392    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch" podUID="9384e565-f55a-45e3-bd08-b86c27972b91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 30 19:10:07.291632 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:07.291531    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:07.496727 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:07.496689    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:07.695749 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:07.695714    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9"]
Oct 30 19:10:07.700053 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:07.700029    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9"
Oct 30 19:10:07.714340 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:07.714309    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:07.755601 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:07.755560    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9"]
Oct 30 19:10:07.780513 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:07.780477    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/011ebb2b-f670-4df6-aff6-1c419f38cb1d-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9\" (UID: \"011ebb2b-f670-4df6-aff6-1c419f38cb1d\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9"
Oct 30 19:10:07.780661 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:07.780548    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c26wk\" (UniqueName: \"kubernetes.io/projected/011ebb2b-f670-4df6-aff6-1c419f38cb1d-kube-api-access-c26wk\") pod \"payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9\" (UID: \"011ebb2b-f670-4df6-aff6-1c419f38cb1d\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9"
Oct 30 19:10:07.800689 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:07.800647    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-b-5a035d\""
Oct 30 19:10:07.890976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:07.885175    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/011ebb2b-f670-4df6-aff6-1c419f38cb1d-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9\" (UID: \"011ebb2b-f670-4df6-aff6-1c419f38cb1d\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9"
Oct 30 19:10:07.891315 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:07.891287    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c26wk\" (UniqueName: \"kubernetes.io/projected/011ebb2b-f670-4df6-aff6-1c419f38cb1d-kube-api-access-c26wk\") pod \"payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9\" (UID: \"011ebb2b-f670-4df6-aff6-1c419f38cb1d\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9"
Oct 30 19:10:07.893966 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:07.892680    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/011ebb2b-f670-4df6-aff6-1c419f38cb1d-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9\" (UID: \"011ebb2b-f670-4df6-aff6-1c419f38cb1d\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9"
Oct 30 19:10:07.940294 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:07.940261    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:10:07.941361 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:07.941268    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:10:07.942183 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:07.942163    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:10:07.953571 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:07.953506    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c26wk\" (UniqueName: \"kubernetes.io/projected/011ebb2b-f670-4df6-aff6-1c419f38cb1d-kube-api-access-c26wk\") pod \"payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9\" (UID: \"011ebb2b-f670-4df6-aff6-1c419f38cb1d\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9"
Oct 30 19:10:08.028401 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:08.028362    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9"
Oct 30 19:10:08.210322 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:10:08.210226    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:10:08.406710 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:08.406684    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9"]
Oct 30 19:10:08.408348 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:10:08.408325    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod011ebb2b_f670_4df6_aff6_1c419f38cb1d.slice/crio-b07398b702da993c5df7f6ef9efd95c6e99c0fee13a317eafaae6feb202599d6 WatchSource:0}: Error finding container b07398b702da993c5df7f6ef9efd95c6e99c0fee13a317eafaae6feb202599d6: Status 404 returned error can't find the container with id b07398b702da993c5df7f6ef9efd95c6e99c0fee13a317eafaae6feb202599d6
Oct 30 19:10:09.312637 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:09.312601    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9" event={"ID":"011ebb2b-f670-4df6-aff6-1c419f38cb1d","Type":"ContainerStarted","Data":"73ad5c5e76c94528797bb58054c4a35dffe6a6f0d8d8c94b125a97877693ce3b"}
Oct 30 19:10:09.313046 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:09.312665    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9" event={"ID":"011ebb2b-f670-4df6-aff6-1c419f38cb1d","Type":"ContainerStarted","Data":"b07398b702da993c5df7f6ef9efd95c6e99c0fee13a317eafaae6feb202599d6"}
Oct 30 19:10:09.313046 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:09.312875    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9"
Oct 30 19:10:09.432636 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:09.432576    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9" podStartSLOduration=2.43255933 podStartE2EDuration="2.43255933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:10:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:10:09.429196145 +0000 UTC m=+2130.140944843" watchObservedRunningTime="2026-10-30 19:10:09.43255933 +0000 UTC m=+2130.144308028"
Oct 30 19:10:12.269859 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:12.269831    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr"
Oct 30 19:10:12.563808 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:12.563755    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:17.402973 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:17.399163    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:20.332025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:20.331992    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9"
Oct 30 19:10:22.226564 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:22.226526    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:23.516053 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:23.514827    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:24.440663 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:24.440626    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:24.617400 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:24.617363    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:27.692813 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:27.692772    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:34.215571 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:34.215515    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:36.521044 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:36.521009    5820 generic.go:356] "Generic (PLEG): container finished" podID="223cde4d-b792-432f-80ca-fb0189521a71" containerID="60d15b8bdd9eb7c0a539a8d15b12e2b971d4d6f31191180766b0eee8eed16b30" exitCode=0
Oct 30 19:10:36.521477 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:36.521050    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t" event={"ID":"223cde4d-b792-432f-80ca-fb0189521a71","Type":"ContainerDied","Data":"60d15b8bdd9eb7c0a539a8d15b12e2b971d4d6f31191180766b0eee8eed16b30"}
Oct 30 19:10:36.521477 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:36.521410    5820 scope.go:122] "RemoveContainer" containerID="60d15b8bdd9eb7c0a539a8d15b12e2b971d4d6f31191180766b0eee8eed16b30"
Oct 30 19:10:36.522970 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:36.522921    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:10:36.523039 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:36.523009    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:10:36.523079 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:36.523043    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:10:37.529824 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:37.529788    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t" event={"ID":"223cde4d-b792-432f-80ca-fb0189521a71","Type":"ContainerStarted","Data":"d11e3e38d1e67d79be444ea8cf05e3d0f6f1edcc9fbf4f64cb8395cb7444a413"}
Oct 30 19:10:37.939516 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:37.939486    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:10:37.957071 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:37.956089    5820 patch_prober.go:29] interesting pod/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:10:37.957355 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:37.957327    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t" podUID="223cde4d-b792-432f-80ca-fb0189521a71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 19:10:38.203740 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:10:38.203658    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:10:38.262997 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:38.262937    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"]
Oct 30 19:10:38.263804 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:38.263471    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2" podUID="7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39" containerName="maas-api" containerID="cri-o://ded21665c7125e94dabe5f7ded56cea8e08bd2322bb1b0b74f66fcc0295eb742" gracePeriod=30
Oct 30 19:10:38.401121 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:38.401088    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:38.567854 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:38.567814    5820 generic.go:356] "Generic (PLEG): container finished" podID="7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39" containerID="ded21665c7125e94dabe5f7ded56cea8e08bd2322bb1b0b74f66fcc0295eb742" exitCode=0
Oct 30 19:10:38.568285 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:38.567997    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2" event={"ID":"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39","Type":"ContainerDied","Data":"ded21665c7125e94dabe5f7ded56cea8e08bd2322bb1b0b74f66fcc0295eb742"}
Oct 30 19:10:38.729031 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:38.722227    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"
Oct 30 19:10:38.794420 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:38.794339    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-maas-api-tls\") pod \"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39\" (UID: \"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39\") "
Oct 30 19:10:38.794571 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:38.794428    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-metrics-certs\") pod \"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39\" (UID: \"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39\") "
Oct 30 19:10:38.794571 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:38.794533    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-openshift-service-ca\") pod \"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39\" (UID: \"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39\") "
Oct 30 19:10:38.794646 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:38.794571    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-kube-api-access-sr858\" (UniqueName: \"kubernetes.io/projected/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-kube-api-access-sr858\") pod \"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39\" (UID: \"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39\") "
Oct 30 19:10:38.795346 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:38.795320    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-openshift-service-ca" pod "7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39" (UID: "7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:10:38.797244 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:38.797212    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-maas-api-tls" pod "7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39" (UID: "7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:10:38.797364 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:38.797285    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-kube-api-access-sr858" pod "7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39" (UID: "7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39"). InnerVolumeSpecName "kube-api-access-sr858". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:10:38.797364 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:38.797319    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-metrics-certs" pod "7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39" (UID: "7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:10:38.897829 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:38.897792    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:10:38.897829 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:38.897831    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:10:38.898053 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:38.897846    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:10:38.898053 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:38.897860    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sr858\" (UniqueName: \"kubernetes.io/projected/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39-kube-api-access-sr858\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:10:38.906202 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:38.906164    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:38.943031 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:38.942333    5820 patch_prober.go:29] interesting pod/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:10:38.943031 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:38.942393    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t" podUID="223cde4d-b792-432f-80ca-fb0189521a71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 19:10:39.130807 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:39.130772    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:39.582160 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:39.582122    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2" event={"ID":"7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39","Type":"ContainerDied","Data":"25e910a065ea4c70ccf2ddbdfed2b02f75798b720981d58b4fb5ef274e88fe33"}
Oct 30 19:10:39.582773 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:39.582739    5820 scope.go:122] "RemoveContainer" containerID="ded21665c7125e94dabe5f7ded56cea8e08bd2322bb1b0b74f66fcc0295eb742"
Oct 30 19:10:39.582983 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:39.582742    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"
Oct 30 19:10:39.770130 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:39.770042    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"]
Oct 30 19:10:39.820054 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:39.820025    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1acf9b-75c5c5d9d8-kw5c2"]
Oct 30 19:10:39.959340 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:39.959306    5820 patch_prober.go:29] interesting pod/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:10:39.959547 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:39.959378    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t" podUID="223cde4d-b792-432f-80ca-fb0189521a71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 19:10:39.968849 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:39.968816    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:40.205057 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:40.205026    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:40.382105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:40.382065    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:40.896015 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:40.895980    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:40.958977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:40.952516    5820 patch_prober.go:29] interesting pod/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:10:40.958977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:40.952589    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t" podUID="223cde4d-b792-432f-80ca-fb0189521a71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 19:10:41.457974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:41.453227    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39" path="/var/lib/kubelet/pods/7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39/volumes"
Oct 30 19:10:41.959977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:41.959559    5820 patch_prober.go:29] interesting pod/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:10:41.959977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:41.959636    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t" podUID="223cde4d-b792-432f-80ca-fb0189521a71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 19:10:43.045892 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:43.045858    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:10:43.046349 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:43.046245    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:10:43.047336 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:43.047315    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:10:45.282448 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:45.282402    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:47.471416 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:47.471328    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:47.696981 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:47.696927    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:47.794089 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:47.793999    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"]
Oct 30 19:10:47.794260 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:47.794174    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp" podUID="3fd20f13-a79c-4b16-b85c-56827c10e159" containerName="istio-proxy" containerID="cri-o://e39b7b83b408119cde6355c962503762bc8177278adcc106402767d9425f1ec8" gracePeriod=30
Oct 30 19:10:47.887578 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:47.885274    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:50.179020 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.178977    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"]
Oct 30 19:10:50.179614 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.179600    5820 cpu_manager.go:414] "containerMap: removing container" podUID="7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39" containerName="maas-api"
Oct 30 19:10:50.179663 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.179614    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39" containerName="maas-api"
Oct 30 19:10:50.179737 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.179728    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c9fcc80-c3b0-4c22-a0f0-a3e405adcf39" containerName="maas-api"
Oct 30 19:10:50.184446 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.184425    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"
Oct 30 19:10:50.249412 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.249379    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"]
Oct 30 19:10:50.275000 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.274935    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8201f1d6-e2ee-493b-81f1-77081a4d4db8-maas-api-tls\") pod \"maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds\" (UID: \"8201f1d6-e2ee-493b-81f1-77081a4d4db8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"
Oct 30 19:10:50.275189 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.275108    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8201f1d6-e2ee-493b-81f1-77081a4d4db8-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds\" (UID: \"8201f1d6-e2ee-493b-81f1-77081a4d4db8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"
Oct 30 19:10:50.275276 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.275256    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8201f1d6-e2ee-493b-81f1-77081a4d4db8-metrics-certs\") pod \"maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds\" (UID: \"8201f1d6-e2ee-493b-81f1-77081a4d4db8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"
Oct 30 19:10:50.275425 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.275401    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s2hp\" (UniqueName: \"kubernetes.io/projected/8201f1d6-e2ee-493b-81f1-77081a4d4db8-kube-api-access-8s2hp\") pod \"maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds\" (UID: \"8201f1d6-e2ee-493b-81f1-77081a4d4db8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"
Oct 30 19:10:50.295025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.294985    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-0be4a4-serving-cert\""
Oct 30 19:10:50.377301 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.377263    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s2hp\" (UniqueName: \"kubernetes.io/projected/8201f1d6-e2ee-493b-81f1-77081a4d4db8-kube-api-access-8s2hp\") pod \"maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds\" (UID: \"8201f1d6-e2ee-493b-81f1-77081a4d4db8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"
Oct 30 19:10:50.377468 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.377342    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8201f1d6-e2ee-493b-81f1-77081a4d4db8-maas-api-tls\") pod \"maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds\" (UID: \"8201f1d6-e2ee-493b-81f1-77081a4d4db8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"
Oct 30 19:10:50.377468 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.377388    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8201f1d6-e2ee-493b-81f1-77081a4d4db8-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds\" (UID: \"8201f1d6-e2ee-493b-81f1-77081a4d4db8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"
Oct 30 19:10:50.377468 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.377459    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8201f1d6-e2ee-493b-81f1-77081a4d4db8-metrics-certs\") pod \"maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds\" (UID: \"8201f1d6-e2ee-493b-81f1-77081a4d4db8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"
Oct 30 19:10:50.377643 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:10:50.377512    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-serving-cert: secret "maas-api-e2e-praxis-a-0be4a4-serving-cert" not found
Oct 30 19:10:50.377643 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:10:50.377621    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8201f1d6-e2ee-493b-81f1-77081a4d4db8-maas-api-tls podName:8201f1d6-e2ee-493b-81f1-77081a4d4db8 nodeName:}" failed. No retries permitted until 2026-10-30 19:10:50.87759914 +0000 UTC m=+2171.589347828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8201f1d6-e2ee-493b-81f1-77081a4d4db8-maas-api-tls") pod "maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds" (UID: "8201f1d6-e2ee-493b-81f1-77081a4d4db8") : secret "maas-api-e2e-praxis-a-0be4a4-serving-cert" not found
Oct 30 19:10:50.378199 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.378175    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8201f1d6-e2ee-493b-81f1-77081a4d4db8-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds\" (UID: \"8201f1d6-e2ee-493b-81f1-77081a4d4db8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"
Oct 30 19:10:50.380019 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.379997    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8201f1d6-e2ee-493b-81f1-77081a4d4db8-metrics-certs\") pod \"maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds\" (UID: \"8201f1d6-e2ee-493b-81f1-77081a4d4db8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"
Oct 30 19:10:50.426444 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.426401    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:50.444516 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.444463    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s2hp\" (UniqueName: \"kubernetes.io/projected/8201f1d6-e2ee-493b-81f1-77081a4d4db8-kube-api-access-8s2hp\") pod \"maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds\" (UID: \"8201f1d6-e2ee-493b-81f1-77081a4d4db8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"
Oct 30 19:10:50.663451 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.663420    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:50.858917 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.858882    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:50.885157 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.885130    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8201f1d6-e2ee-493b-81f1-77081a4d4db8-maas-api-tls\") pod \"maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds\" (UID: \"8201f1d6-e2ee-493b-81f1-77081a4d4db8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"
Oct 30 19:10:50.887631 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:50.887604    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8201f1d6-e2ee-493b-81f1-77081a4d4db8-maas-api-tls\") pod \"maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds\" (UID: \"8201f1d6-e2ee-493b-81f1-77081a4d4db8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"
Oct 30 19:10:51.100368 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:51.100333    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"
Oct 30 19:10:51.481478 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:51.481447    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"]
Oct 30 19:10:51.482754 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:10:51.482704    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8201f1d6_e2ee_493b_81f1_77081a4d4db8.slice/crio-284c0725d674d724bc6bf742d7d19f7b760f1e6375bff0912f0b31e9e24a37ec WatchSource:0}: Error finding container 284c0725d674d724bc6bf742d7d19f7b760f1e6375bff0912f0b31e9e24a37ec: Status 404 returned error can't find the container with id 284c0725d674d724bc6bf742d7d19f7b760f1e6375bff0912f0b31e9e24a37ec
Oct 30 19:10:51.672136 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:51.672105    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds" event={"ID":"8201f1d6-e2ee-493b-81f1-77081a4d4db8","Type":"ContainerStarted","Data":"284c0725d674d724bc6bf742d7d19f7b760f1e6375bff0912f0b31e9e24a37ec"}
Oct 30 19:10:52.679983 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:52.679887    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds" event={"ID":"8201f1d6-e2ee-493b-81f1-77081a4d4db8","Type":"ContainerStarted","Data":"924ff12d171faabfd5a1eede837ed31b00c11f51d78c09403fa7b331fce83d8d"}
Oct 30 19:10:52.680453 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:52.680020    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"
Oct 30 19:10:52.800251 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:52.800192    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds" podStartSLOduration=1.943559907 podStartE2EDuration="2.800179379s" totalImagesPullingTime="856.619472ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:10:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:10:52.79711549 +0000 UTC m=+2173.508864192" watchObservedRunningTime="2026-10-30 19:10:52.800179379 +0000 UTC m=+2173.511928110"
Oct 30 19:10:53.107124 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.107092    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:10:53.149048 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.149016    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3fd20f13-a79c-4b16-b85c-56827c10e159-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fd20f13-a79c-4b16-b85c-56827c10e159-istiod-ca-cert\") pod \"3fd20f13-a79c-4b16-b85c-56827c10e159\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") "
Oct 30 19:10:53.149206 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.149079    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3fd20f13-a79c-4b16-b85c-56827c10e159-kube-api-access-mghzs\" (UniqueName: \"kubernetes.io/projected/3fd20f13-a79c-4b16-b85c-56827c10e159-kube-api-access-mghzs\") pod \"3fd20f13-a79c-4b16-b85c-56827c10e159\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") "
Oct 30 19:10:53.149206 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.149129    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-envoy\") pod \"3fd20f13-a79c-4b16-b85c-56827c10e159\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") "
Oct 30 19:10:53.149206 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.149199    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-podinfo\") pod \"3fd20f13-a79c-4b16-b85c-56827c10e159\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") "
Oct 30 19:10:53.149379 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.149229    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-workload-certs\") pod \"3fd20f13-a79c-4b16-b85c-56827c10e159\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") "
Oct 30 19:10:53.149379 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.149347    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-credential-socket\") pod \"3fd20f13-a79c-4b16-b85c-56827c10e159\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") "
Oct 30 19:10:53.149490 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.149403    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-token\" (UniqueName: \"kubernetes.io/projected/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-token\") pod \"3fd20f13-a79c-4b16-b85c-56827c10e159\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") "
Oct 30 19:10:53.149490 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.149407    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fd20f13-a79c-4b16-b85c-56827c10e159-istiod-ca-cert" pod "3fd20f13-a79c-4b16-b85c-56827c10e159" (UID: "3fd20f13-a79c-4b16-b85c-56827c10e159"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:10:53.149610 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.149528    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-workload-socket\") pod \"3fd20f13-a79c-4b16-b85c-56827c10e159\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") "
Oct 30 19:10:53.149610 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.149556    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-workload-certs" pod "3fd20f13-a79c-4b16-b85c-56827c10e159" (UID: "3fd20f13-a79c-4b16-b85c-56827c10e159"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:10:53.149610 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.149560    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-data\") pod \"3fd20f13-a79c-4b16-b85c-56827c10e159\" (UID: \"3fd20f13-a79c-4b16-b85c-56827c10e159\") "
Oct 30 19:10:53.149786 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.149767    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-data" pod "3fd20f13-a79c-4b16-b85c-56827c10e159" (UID: "3fd20f13-a79c-4b16-b85c-56827c10e159"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:10:53.149846 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.149780    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-workload-socket" pod "3fd20f13-a79c-4b16-b85c-56827c10e159" (UID: "3fd20f13-a79c-4b16-b85c-56827c10e159"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:10:53.150531 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.150021    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-credential-socket" pod "3fd20f13-a79c-4b16-b85c-56827c10e159" (UID: "3fd20f13-a79c-4b16-b85c-56827c10e159"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:10:53.150531 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.150434    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:10:53.150531 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.150456    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:10:53.150531 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.150472    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fd20f13-a79c-4b16-b85c-56827c10e159-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:10:53.150531 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.150486    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:10:53.150531 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.150502    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:10:53.151623 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.151592    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-podinfo" pod "3fd20f13-a79c-4b16-b85c-56827c10e159" (UID: "3fd20f13-a79c-4b16-b85c-56827c10e159"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:10:53.151623 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.151605    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-envoy" pod "3fd20f13-a79c-4b16-b85c-56827c10e159" (UID: "3fd20f13-a79c-4b16-b85c-56827c10e159"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:10:53.151797 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.151740    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fd20f13-a79c-4b16-b85c-56827c10e159-kube-api-access-mghzs" pod "3fd20f13-a79c-4b16-b85c-56827c10e159" (UID: "3fd20f13-a79c-4b16-b85c-56827c10e159"). InnerVolumeSpecName "kube-api-access-mghzs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:10:53.151797 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.151784    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-token" pod "3fd20f13-a79c-4b16-b85c-56827c10e159" (UID: "3fd20f13-a79c-4b16-b85c-56827c10e159"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:10:53.252305 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.252271    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mghzs\" (UniqueName: \"kubernetes.io/projected/3fd20f13-a79c-4b16-b85c-56827c10e159-kube-api-access-mghzs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:10:53.252305 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.252308    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:10:53.252570 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.252344    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:10:53.252570 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.252361    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fd20f13-a79c-4b16-b85c-56827c10e159-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:10:53.538992 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.538962    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:53.688579 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.688542    5820 generic.go:356] "Generic (PLEG): container finished" podID="3fd20f13-a79c-4b16-b85c-56827c10e159" containerID="e39b7b83b408119cde6355c962503762bc8177278adcc106402767d9425f1ec8" exitCode=0
Oct 30 19:10:53.689028 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.688600    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"
Oct 30 19:10:53.689028 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.688621    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp" event={"ID":"3fd20f13-a79c-4b16-b85c-56827c10e159","Type":"ContainerDied","Data":"e39b7b83b408119cde6355c962503762bc8177278adcc106402767d9425f1ec8"}
Oct 30 19:10:53.689028 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.688701    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp" event={"ID":"3fd20f13-a79c-4b16-b85c-56827c10e159","Type":"ContainerDied","Data":"4445ccc30cdcc4b14bd90fada8f7075816831c3bcdb5c9e864e55df63bea6bd7"}
Oct 30 19:10:53.689028 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.688731    5820 scope.go:122] "RemoveContainer" containerID="e39b7b83b408119cde6355c962503762bc8177278adcc106402767d9425f1ec8"
Oct 30 19:10:53.699168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.699134    5820 scope.go:122] "RemoveContainer" containerID="454a8da06583e19d770cffddac86965d58f09eddd5ccf031fcf3cb3f7ee0c243"
Oct 30 19:10:53.708022 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.708006    5820 scope.go:122] "RemoveContainer" containerID="e39b7b83b408119cde6355c962503762bc8177278adcc106402767d9425f1ec8"
Oct 30 19:10:53.708272 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:10:53.708248    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e39b7b83b408119cde6355c962503762bc8177278adcc106402767d9425f1ec8\": container with ID starting with e39b7b83b408119cde6355c962503762bc8177278adcc106402767d9425f1ec8 not found: ID does not exist" containerID="e39b7b83b408119cde6355c962503762bc8177278adcc106402767d9425f1ec8"
Oct 30 19:10:53.708330 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.708274    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e39b7b83b408119cde6355c962503762bc8177278adcc106402767d9425f1ec8"} err="failed to get container status \"e39b7b83b408119cde6355c962503762bc8177278adcc106402767d9425f1ec8\": rpc error: code = NotFound desc = could not find container \"e39b7b83b408119cde6355c962503762bc8177278adcc106402767d9425f1ec8\": container with ID starting with e39b7b83b408119cde6355c962503762bc8177278adcc106402767d9425f1ec8 not found: ID does not exist"
Oct 30 19:10:53.708330 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.708294    5820 scope.go:122] "RemoveContainer" containerID="454a8da06583e19d770cffddac86965d58f09eddd5ccf031fcf3cb3f7ee0c243"
Oct 30 19:10:53.708545 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:10:53.708525    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"454a8da06583e19d770cffddac86965d58f09eddd5ccf031fcf3cb3f7ee0c243\": container with ID starting with 454a8da06583e19d770cffddac86965d58f09eddd5ccf031fcf3cb3f7ee0c243 not found: ID does not exist" containerID="454a8da06583e19d770cffddac86965d58f09eddd5ccf031fcf3cb3f7ee0c243"
Oct 30 19:10:53.708588 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.708552    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"454a8da06583e19d770cffddac86965d58f09eddd5ccf031fcf3cb3f7ee0c243"} err="failed to get container status \"454a8da06583e19d770cffddac86965d58f09eddd5ccf031fcf3cb3f7ee0c243\": rpc error: code = NotFound desc = could not find container \"454a8da06583e19d770cffddac86965d58f09eddd5ccf031fcf3cb3f7ee0c243\": container with ID starting with 454a8da06583e19d770cffddac86965d58f09eddd5ccf031fcf3cb3f7ee0c243 not found: ID does not exist"
Oct 30 19:10:53.864261 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.864228    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"]
Oct 30 19:10:53.913551 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:53.913524    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-1acf9b-openshift-default-f76b4d675-cnmjp"]
Oct 30 19:10:55.153622 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:55.153584    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:55.328686 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:55.328650    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:55.436663 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:55.436591    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3fd20f13-a79c-4b16-b85c-56827c10e159" path="/var/lib/kubelet/pods/3fd20f13-a79c-4b16-b85c-56827c10e159/volumes"
Oct 30 19:10:55.731322 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:55.727724    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:57.317080 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:57.317058    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-7kxzm_88b8cb4c-656b-44e4-8632-0acff65c58ad/payload-processing/0.log"
Oct 30 19:10:57.580874 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:57.580796    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-7kxzm_88b8cb4c-656b-44e4-8632-0acff65c58ad/payload-processing/0.log"
Oct 30 19:10:58.669457 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:58.669423    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:58.694388 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:10:58.694361    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"
Oct 30 19:11:00.617849 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:00.617804    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:00.816974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:00.815519    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:02.037527 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:02.037496    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:05.742034 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:05.739053    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:08.203299 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:11:08.203262    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:11:10.505531 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:10.505492    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:15.319865 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:15.319735    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:19.529254 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:19.529212    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:21.148214 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:21.148171    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:21.396013 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:21.395971    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:21.573871 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:21.573835    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:22.106654 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:22.106614    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:25.695874 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:25.695841    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"]
Oct 30 19:11:25.696389 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:25.696068    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk" podUID="d5ef49d3-3505-4ae5-aa39-ddeebffa35fd" containerName="maas-api" containerID="cri-o://fe8107be5499c32a293ca5bea7078e3c1c9862bb226bea0dc4d72e2ed4b54de2" gracePeriod=30
Oct 30 19:11:25.834299 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:25.834257    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:25.956063 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:25.955765    5820 generic.go:356] "Generic (PLEG): container finished" podID="d5ef49d3-3505-4ae5-aa39-ddeebffa35fd" containerID="fe8107be5499c32a293ca5bea7078e3c1c9862bb226bea0dc4d72e2ed4b54de2" exitCode=0
Oct 30 19:11:25.956063 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:25.955909    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk" event={"ID":"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd","Type":"ContainerDied","Data":"fe8107be5499c32a293ca5bea7078e3c1c9862bb226bea0dc4d72e2ed4b54de2"}
Oct 30 19:11:26.002512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:26.002486    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"
Oct 30 19:11:26.050269 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:26.050238    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-metrics-certs\") pod \"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd\" (UID: \"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd\") "
Oct 30 19:11:26.050448 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:26.050363    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-kube-api-access-q2xdn\" (UniqueName: \"kubernetes.io/projected/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-kube-api-access-q2xdn\") pod \"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd\" (UID: \"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd\") "
Oct 30 19:11:26.050448 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:26.050431    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-openshift-service-ca\") pod \"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd\" (UID: \"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd\") "
Oct 30 19:11:26.050582 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:26.050536    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-maas-api-tls\") pod \"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd\" (UID: \"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd\") "
Oct 30 19:11:26.050789 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:26.050757    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-openshift-service-ca" pod "d5ef49d3-3505-4ae5-aa39-ddeebffa35fd" (UID: "d5ef49d3-3505-4ae5-aa39-ddeebffa35fd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:11:26.051128 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:26.051035    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:26.052644 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:26.052622    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-metrics-certs" pod "d5ef49d3-3505-4ae5-aa39-ddeebffa35fd" (UID: "d5ef49d3-3505-4ae5-aa39-ddeebffa35fd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:11:26.052644 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:26.052626    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-kube-api-access-q2xdn" pod "d5ef49d3-3505-4ae5-aa39-ddeebffa35fd" (UID: "d5ef49d3-3505-4ae5-aa39-ddeebffa35fd"). InnerVolumeSpecName "kube-api-access-q2xdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:11:26.052824 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:26.052669    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-maas-api-tls" pod "d5ef49d3-3505-4ae5-aa39-ddeebffa35fd" (UID: "d5ef49d3-3505-4ae5-aa39-ddeebffa35fd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:11:26.153973 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:26.153135    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:26.153973 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:26.153175    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:26.153973 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:26.153191    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2xdn\" (UniqueName: \"kubernetes.io/projected/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd-kube-api-access-q2xdn\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:26.970837 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:26.970627    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk" event={"ID":"d5ef49d3-3505-4ae5-aa39-ddeebffa35fd","Type":"ContainerDied","Data":"85b3a0e92070df3cbe4955b572e6a21df07fb6627624478315ef0c250ae5eff4"}
Oct 30 19:11:26.970837 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:26.970708    5820 scope.go:122] "RemoveContainer" containerID="fe8107be5499c32a293ca5bea7078e3c1c9862bb226bea0dc4d72e2ed4b54de2"
Oct 30 19:11:26.971392 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:26.970885    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"
Oct 30 19:11:27.171668 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:27.171627    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"]
Oct 30 19:11:27.246021 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:27.245135    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e67448-85c59dbb5d-x2vqk"]
Oct 30 19:11:27.460312 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:27.460272    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d5ef49d3-3505-4ae5-aa39-ddeebffa35fd" path="/var/lib/kubelet/pods/d5ef49d3-3505-4ae5-aa39-ddeebffa35fd/volumes"
Oct 30 19:11:27.560134 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:27.560086    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:27.862030 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:27.861159    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:28.053142 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:28.053091    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:28.476898 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:28.476851    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:29.287671 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:29.287618    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:29.499914 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:29.499878    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:29.689213 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:29.689176    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"]
Oct 30 19:11:29.689414 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:29.689391    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb" podUID="99f1797d-6313-4c12-a997-2c70b1c70d92" containerName="istio-proxy" containerID="cri-o://41e6824f7c468347dc036a30bc866b6609deefdb899a236c50da5ab28abfa4e6" gracePeriod=30
Oct 30 19:11:29.762003 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:29.761550    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:31.257564 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:31.257525    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:32.089582 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:32.089541    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:32.397135 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:32.397052    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:32.970536 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:32.970506    5820 patch_prober.go:29] interesting pod/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:11:32.970690 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:32.970576    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb" podUID="99f1797d-6313-4c12-a997-2c70b1c70d92" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:11:34.870592 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:34.870551    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:34.990016 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:34.989992    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:11:35.031406 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.031369    5820 generic.go:356] "Generic (PLEG): container finished" podID="99f1797d-6313-4c12-a997-2c70b1c70d92" containerID="41e6824f7c468347dc036a30bc866b6609deefdb899a236c50da5ab28abfa4e6" exitCode=0
Oct 30 19:11:35.031572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.031471    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb" event={"ID":"99f1797d-6313-4c12-a997-2c70b1c70d92","Type":"ContainerDied","Data":"41e6824f7c468347dc036a30bc866b6609deefdb899a236c50da5ab28abfa4e6"}
Oct 30 19:11:35.031572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.031523    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb" event={"ID":"99f1797d-6313-4c12-a997-2c70b1c70d92","Type":"ContainerDied","Data":"726ad4d4af90264974df0399746d968914cebc88a107397f780f0b0ec1fe374e"}
Oct 30 19:11:35.031572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.031550    5820 scope.go:122] "RemoveContainer" containerID="41e6824f7c468347dc036a30bc866b6609deefdb899a236c50da5ab28abfa4e6"
Oct 30 19:11:35.031748 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.031702    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"
Oct 30 19:11:35.073332 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.073206    5820 scope.go:122] "RemoveContainer" containerID="8573b112d9779ec6306a4dcc10578c21f9b0f2a0c7f3bff6aff99a758b1f5d22"
Oct 30 19:11:35.073719 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.073688    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-token\" (UniqueName: \"kubernetes.io/projected/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-token\") pod \"99f1797d-6313-4c12-a997-2c70b1c70d92\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") "
Oct 30 19:11:35.073847 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.073768    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-workload-certs\") pod \"99f1797d-6313-4c12-a997-2c70b1c70d92\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") "
Oct 30 19:11:35.073847 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.073837    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-envoy\") pod \"99f1797d-6313-4c12-a997-2c70b1c70d92\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") "
Oct 30 19:11:35.074094 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.073888    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99f1797d-6313-4c12-a997-2c70b1c70d92-kube-api-access-wmcps\" (UniqueName: \"kubernetes.io/projected/99f1797d-6313-4c12-a997-2c70b1c70d92-kube-api-access-wmcps\") pod \"99f1797d-6313-4c12-a997-2c70b1c70d92\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") "
Oct 30 19:11:35.074094 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.073933    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99f1797d-6313-4c12-a997-2c70b1c70d92-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99f1797d-6313-4c12-a997-2c70b1c70d92-istiod-ca-cert\") pod \"99f1797d-6313-4c12-a997-2c70b1c70d92\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") "
Oct 30 19:11:35.074094 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.074036    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-data\") pod \"99f1797d-6313-4c12-a997-2c70b1c70d92\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") "
Oct 30 19:11:35.074265 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.074131    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-podinfo\") pod \"99f1797d-6313-4c12-a997-2c70b1c70d92\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") "
Oct 30 19:11:35.077768 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.075752    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-credential-socket\") pod \"99f1797d-6313-4c12-a997-2c70b1c70d92\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") "
Oct 30 19:11:35.077768 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.075844    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-workload-socket\") pod \"99f1797d-6313-4c12-a997-2c70b1c70d92\" (UID: \"99f1797d-6313-4c12-a997-2c70b1c70d92\") "
Oct 30 19:11:35.077768 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.076163    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-workload-certs" pod "99f1797d-6313-4c12-a997-2c70b1c70d92" (UID: "99f1797d-6313-4c12-a997-2c70b1c70d92"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:11:35.077768 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.076626    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-workload-socket" pod "99f1797d-6313-4c12-a997-2c70b1c70d92" (UID: "99f1797d-6313-4c12-a997-2c70b1c70d92"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:11:35.077768 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.076637    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:35.077768 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.076645    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99f1797d-6313-4c12-a997-2c70b1c70d92-istiod-ca-cert" pod "99f1797d-6313-4c12-a997-2c70b1c70d92" (UID: "99f1797d-6313-4c12-a997-2c70b1c70d92"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:11:35.077768 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.076852    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-data" pod "99f1797d-6313-4c12-a997-2c70b1c70d92" (UID: "99f1797d-6313-4c12-a997-2c70b1c70d92"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:11:35.080377 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.078611    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:35.080377 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.079299    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-credential-socket" pod "99f1797d-6313-4c12-a997-2c70b1c70d92" (UID: "99f1797d-6313-4c12-a997-2c70b1c70d92"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:11:35.082610 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.082574    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-envoy" pod "99f1797d-6313-4c12-a997-2c70b1c70d92" (UID: "99f1797d-6313-4c12-a997-2c70b1c70d92"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:11:35.082877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.082852    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-token" pod "99f1797d-6313-4c12-a997-2c70b1c70d92" (UID: "99f1797d-6313-4c12-a997-2c70b1c70d92"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:11:35.094553 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.094516    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-podinfo" pod "99f1797d-6313-4c12-a997-2c70b1c70d92" (UID: "99f1797d-6313-4c12-a997-2c70b1c70d92"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:11:35.095594 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.095570    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99f1797d-6313-4c12-a997-2c70b1c70d92-kube-api-access-wmcps" pod "99f1797d-6313-4c12-a997-2c70b1c70d92" (UID: "99f1797d-6313-4c12-a997-2c70b1c70d92"). InnerVolumeSpecName "kube-api-access-wmcps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:11:35.115584 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.115557    5820 scope.go:122] "RemoveContainer" containerID="41e6824f7c468347dc036a30bc866b6609deefdb899a236c50da5ab28abfa4e6"
Oct 30 19:11:35.116199 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:11:35.116054    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41e6824f7c468347dc036a30bc866b6609deefdb899a236c50da5ab28abfa4e6\": container with ID starting with 41e6824f7c468347dc036a30bc866b6609deefdb899a236c50da5ab28abfa4e6 not found: ID does not exist" containerID="41e6824f7c468347dc036a30bc866b6609deefdb899a236c50da5ab28abfa4e6"
Oct 30 19:11:35.116199 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.116097    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41e6824f7c468347dc036a30bc866b6609deefdb899a236c50da5ab28abfa4e6"} err="failed to get container status \"41e6824f7c468347dc036a30bc866b6609deefdb899a236c50da5ab28abfa4e6\": rpc error: code = NotFound desc = could not find container \"41e6824f7c468347dc036a30bc866b6609deefdb899a236c50da5ab28abfa4e6\": container with ID starting with 41e6824f7c468347dc036a30bc866b6609deefdb899a236c50da5ab28abfa4e6 not found: ID does not exist"
Oct 30 19:11:35.116199 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.116121    5820 scope.go:122] "RemoveContainer" containerID="8573b112d9779ec6306a4dcc10578c21f9b0f2a0c7f3bff6aff99a758b1f5d22"
Oct 30 19:11:35.116870 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:11:35.116811    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8573b112d9779ec6306a4dcc10578c21f9b0f2a0c7f3bff6aff99a758b1f5d22\": container with ID starting with 8573b112d9779ec6306a4dcc10578c21f9b0f2a0c7f3bff6aff99a758b1f5d22 not found: ID does not exist" containerID="8573b112d9779ec6306a4dcc10578c21f9b0f2a0c7f3bff6aff99a758b1f5d22"
Oct 30 19:11:35.116870 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.116841    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8573b112d9779ec6306a4dcc10578c21f9b0f2a0c7f3bff6aff99a758b1f5d22"} err="failed to get container status \"8573b112d9779ec6306a4dcc10578c21f9b0f2a0c7f3bff6aff99a758b1f5d22\": rpc error: code = NotFound desc = could not find container \"8573b112d9779ec6306a4dcc10578c21f9b0f2a0c7f3bff6aff99a758b1f5d22\": container with ID starting with 8573b112d9779ec6306a4dcc10578c21f9b0f2a0c7f3bff6aff99a758b1f5d22 not found: ID does not exist"
Oct 30 19:11:35.150986 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.150916    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"]
Oct 30 19:11:35.151230 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.151206    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98" podUID="cda46604-6d15-4ee9-823f-cb5cb70f4c51" containerName="istio-proxy" containerID="cri-o://b437d90c6fa5ae131fb8ae9af8bf2ca377a3d06e4799dac75ee77ba890e48e82" gracePeriod=30
Oct 30 19:11:35.178234 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.178195    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:35.178234 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.178228    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:35.178517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.178243    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:35.178517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.178257    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmcps\" (UniqueName: \"kubernetes.io/projected/99f1797d-6313-4c12-a997-2c70b1c70d92-kube-api-access-wmcps\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:35.178517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.178271    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99f1797d-6313-4c12-a997-2c70b1c70d92-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:35.178517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.178289    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:35.178517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.178302    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99f1797d-6313-4c12-a997-2c70b1c70d92-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:35.178517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.178316    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99f1797d-6313-4c12-a997-2c70b1c70d92-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:35.302030 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.280529    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:35.535882 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.535851    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"]
Oct 30 19:11:35.538413 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:35.538392    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-4ebf78-openshift-default-646b9fdb77-lvhmb"]
Oct 30 19:11:36.178654 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:36.178616    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:36.804868 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:36.804839    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:37.017774 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.017699    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:37.047300 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.047225    5820 generic.go:356] "Generic (PLEG): container finished" podID="4a0996ab-5aaa-4b71-b064-87d9d1572d13" containerID="fa6c9f30b0bb40402b99756c717058cc797c78f47ad97cb081f4876bd879ea3c" exitCode=0
Oct 30 19:11:37.047300 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.047263    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6" event={"ID":"4a0996ab-5aaa-4b71-b064-87d9d1572d13","Type":"ContainerDied","Data":"fa6c9f30b0bb40402b99756c717058cc797c78f47ad97cb081f4876bd879ea3c"}
Oct 30 19:11:37.047676 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.047660    5820 scope.go:122] "RemoveContainer" containerID="fa6c9f30b0bb40402b99756c717058cc797c78f47ad97cb081f4876bd879ea3c"
Oct 30 19:11:37.049169 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.049138    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:11:37.049247 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.049211    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:11:37.049284 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.049248    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:11:37.166639 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.166606    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:11:37.241575 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.241543    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"]
Oct 30 19:11:37.242317 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.242298    5820 cpu_manager.go:414] "containerMap: removing container" podUID="3fd20f13-a79c-4b16-b85c-56827c10e159" containerName="istio-proxy"
Oct 30 19:11:37.242379 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.242319    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3fd20f13-a79c-4b16-b85c-56827c10e159" containerName="istio-proxy"
Oct 30 19:11:37.242379 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.242349    5820 cpu_manager.go:414] "containerMap: removing container" podUID="d5ef49d3-3505-4ae5-aa39-ddeebffa35fd" containerName="maas-api"
Oct 30 19:11:37.242379 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.242357    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5ef49d3-3505-4ae5-aa39-ddeebffa35fd" containerName="maas-api"
Oct 30 19:11:37.242379 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.242366    5820 cpu_manager.go:414] "containerMap: removing container" podUID="99f1797d-6313-4c12-a997-2c70b1c70d92" containerName="istio-proxy"
Oct 30 19:11:37.242379 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.242374    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99f1797d-6313-4c12-a997-2c70b1c70d92" containerName="istio-proxy"
Oct 30 19:11:37.242619 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.242488    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="3fd20f13-a79c-4b16-b85c-56827c10e159" containerName="istio-proxy"
Oct 30 19:11:37.242619 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.242502    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5ef49d3-3505-4ae5-aa39-ddeebffa35fd" containerName="maas-api"
Oct 30 19:11:37.242619 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.242509    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="99f1797d-6313-4c12-a997-2c70b1c70d92" containerName="istio-proxy"
Oct 30 19:11:37.242619 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.242517    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="99f1797d-6313-4c12-a997-2c70b1c70d92" containerName="istio-proxy"
Oct 30 19:11:37.245051 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.245030    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.313225 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.313197    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"]
Oct 30 19:11:37.372478 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.372444    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-52d383-openshift-default-dockercfg-4rvp6\""
Oct 30 19:11:37.418083 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.411244    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:37.418425 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.412981    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-data\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.418620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.418602    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-envoy\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.418732 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.418719    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngd7v\" (UniqueName: \"kubernetes.io/projected/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-kube-api-access-ngd7v\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.418844 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.418827    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istiod-ca-cert\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.419073 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.419052    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-workload-certs\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.419241 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.419222    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-podinfo\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.419381 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.419364    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-credential-socket\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.419511 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.419496    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-token\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.419694 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.419679    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-workload-socket\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.438929 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.438900    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="99f1797d-6313-4c12-a997-2c70b1c70d92" path="/var/lib/kubelet/pods/99f1797d-6313-4c12-a997-2c70b1c70d92/volumes"
Oct 30 19:11:37.520803 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.520767    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-data\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.521047 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.520838    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-envoy\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.521047 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.520876    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngd7v\" (UniqueName: \"kubernetes.io/projected/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-kube-api-access-ngd7v\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.521047 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.520900    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istiod-ca-cert\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.521047 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.520998    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-workload-certs\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.521047 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.521018    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-podinfo\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.521374 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.521058    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-credential-socket\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.521374 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.521101    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-token\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.521374 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.521185    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-workload-socket\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.521374 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.521255    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-data\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.521374 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.521344    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-workload-certs\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.521810 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.521488    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-workload-socket\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.521810 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.521540    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-credential-socket\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.522323 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.522297    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istiod-ca-cert\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.524919 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.523968    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-envoy\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.532451 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.532425    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-podinfo\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.587346 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.584646    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngd7v\" (UniqueName: \"kubernetes.io/projected/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-kube-api-access-ngd7v\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.587549 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.587514    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-token\") pod \"e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") " pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:37.621781 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.621745    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:37.859660 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:37.859553    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:38.068444 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:38.068416    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6" event={"ID":"4a0996ab-5aaa-4b71-b064-87d9d1572d13","Type":"ContainerStarted","Data":"eeafa7b7322daa533729b4543a212ac1f6cc19b8ff9cf0832fd11353f8675506"}
Oct 30 19:11:38.204434 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:11:38.204347    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:11:38.361700 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:38.361662    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"]
Oct 30 19:11:38.364445 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:11:38.364417    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31cb668c_773d_4a1c_8264_a8f1b6f6e9b9.slice/crio-a7a7a3be054e37ae3d850fb24380594c9a798b6f7857341f4420b1436175f957 WatchSource:0}: Error finding container a7a7a3be054e37ae3d850fb24380594c9a798b6f7857341f4420b1436175f957: Status 404 returned error can't find the container with id a7a7a3be054e37ae3d850fb24380594c9a798b6f7857341f4420b1436175f957
Oct 30 19:11:38.367079 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:38.367048    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:11:38.367164 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:38.367114    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:11:38.367164 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:38.367145    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:11:39.053363 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:39.051237    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:11:39.059083 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:39.059041    5820 patch_prober.go:29] interesting pod/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 19:11:39.059236 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:39.059098    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6" podUID="4a0996ab-5aaa-4b71-b064-87d9d1572d13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 19:11:39.076498 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:39.076468    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx" event={"ID":"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9","Type":"ContainerStarted","Data":"443e08ecf06fa9576b471d7ac215f5aff6534ae8cc9faf58c00847e117a32811"}
Oct 30 19:11:39.076620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:39.076520    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx" event={"ID":"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9","Type":"ContainerStarted","Data":"a7a7a3be054e37ae3d850fb24380594c9a798b6f7857341f4420b1436175f957"}
Oct 30 19:11:39.200205 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:39.200153    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx" podStartSLOduration=2.200139026 podStartE2EDuration="2.200139026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:11:39.196848143 +0000 UTC m=+2219.908596839" watchObservedRunningTime="2026-10-30 19:11:39.200139026 +0000 UTC m=+2219.911887773"
Oct 30 19:11:39.860271 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:39.860241    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:39.866060 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:39.866035    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:40.060008 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.042281    5820 patch_prober.go:29] interesting pod/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 19:11:40.060008 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.042353    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6" podUID="4a0996ab-5aaa-4b71-b064-87d9d1572d13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 19:11:40.084206 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.084169    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:40.085041 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.085018    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:11:40.161031 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.160648    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:40.369978 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.358627    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:40.565675 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.565647    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d"]
Oct 30 19:11:40.566395 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.566372    5820 cpu_manager.go:414] "containerMap: removing container" podUID="3fd20f13-a79c-4b16-b85c-56827c10e159" containerName="istio-proxy"
Oct 30 19:11:40.566520 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.566396    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3fd20f13-a79c-4b16-b85c-56827c10e159" containerName="istio-proxy"
Oct 30 19:11:40.566520 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.566443    5820 cpu_manager.go:414] "containerMap: removing container" podUID="99f1797d-6313-4c12-a997-2c70b1c70d92" containerName="istio-proxy"
Oct 30 19:11:40.566520 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.566453    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99f1797d-6313-4c12-a997-2c70b1c70d92" containerName="istio-proxy"
Oct 30 19:11:40.566671 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.566574    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:40.566671 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.566628    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="3fd20f13-a79c-4b16-b85c-56827c10e159" containerName="istio-proxy"
Oct 30 19:11:40.568882 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.568867    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d"
Oct 30 19:11:40.576476 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.576458    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:11:40.609108 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.609084    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d"]
Oct 30 19:11:40.622996 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.622935    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-a-f554bb\""
Oct 30 19:11:40.623116 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.623013    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-a-f554bb-dockercfg-vqk9l\""
Oct 30 19:11:40.755023 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.754389    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-data\") pod \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") "
Oct 30 19:11:40.755023 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.754469    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-workload-socket\") pod \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") "
Oct 30 19:11:40.755023 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.754547    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-token\" (UniqueName: \"kubernetes.io/projected/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-token\") pod \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") "
Oct 30 19:11:40.762990 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.757253    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-workload-certs" pod "cda46604-6d15-4ee9-823f-cb5cb70f4c51" (UID: "cda46604-6d15-4ee9-823f-cb5cb70f4c51"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:11:40.808903 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.807705    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-data" pod "cda46604-6d15-4ee9-823f-cb5cb70f4c51" (UID: "cda46604-6d15-4ee9-823f-cb5cb70f4c51"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:11:40.808903 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.808159    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-workload-socket" pod "cda46604-6d15-4ee9-823f-cb5cb70f4c51" (UID: "cda46604-6d15-4ee9-823f-cb5cb70f4c51"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:11:40.808903 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.754628    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-workload-certs\") pod \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") "
Oct 30 19:11:40.808903 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.808345    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istiod-ca-cert\") pod \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") "
Oct 30 19:11:40.808903 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.808432    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-envoy\") pod \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") "
Oct 30 19:11:40.808903 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.808497    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-credential-socket\") pod \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") "
Oct 30 19:11:40.808903 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.808537    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-podinfo\") pod \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") "
Oct 30 19:11:40.808903 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.808586    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cda46604-6d15-4ee9-823f-cb5cb70f4c51-kube-api-access-m8kk8\" (UniqueName: \"kubernetes.io/projected/cda46604-6d15-4ee9-823f-cb5cb70f4c51-kube-api-access-m8kk8\") pod \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\" (UID: \"cda46604-6d15-4ee9-823f-cb5cb70f4c51\") "
Oct 30 19:11:40.809571 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.809541    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x55j8\" (UniqueName: \"kubernetes.io/projected/337f5ca4-5214-48b2-a02b-671382ec43b2-kube-api-access-x55j8\") pod \"payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d\" (UID: \"337f5ca4-5214-48b2-a02b-671382ec43b2\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d"
Oct 30 19:11:40.809640 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.809621    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/337f5ca4-5214-48b2-a02b-671382ec43b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d\" (UID: \"337f5ca4-5214-48b2-a02b-671382ec43b2\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d"
Oct 30 19:11:40.810464 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.810431    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:40.810464 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.810465    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:40.810677 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.810655    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:40.811125 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.811094    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-token" pod "cda46604-6d15-4ee9-823f-cb5cb70f4c51" (UID: "cda46604-6d15-4ee9-823f-cb5cb70f4c51"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:11:40.811237 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.811209    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istiod-ca-cert" pod "cda46604-6d15-4ee9-823f-cb5cb70f4c51" (UID: "cda46604-6d15-4ee9-823f-cb5cb70f4c51"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:11:40.820044 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.820006    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-credential-socket" pod "cda46604-6d15-4ee9-823f-cb5cb70f4c51" (UID: "cda46604-6d15-4ee9-823f-cb5cb70f4c51"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:11:40.840982 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.828130    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-envoy" pod "cda46604-6d15-4ee9-823f-cb5cb70f4c51" (UID: "cda46604-6d15-4ee9-823f-cb5cb70f4c51"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:11:40.840982 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.831421    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cda46604-6d15-4ee9-823f-cb5cb70f4c51-kube-api-access-m8kk8" pod "cda46604-6d15-4ee9-823f-cb5cb70f4c51" (UID: "cda46604-6d15-4ee9-823f-cb5cb70f4c51"). InnerVolumeSpecName "kube-api-access-m8kk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:11:40.840982 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.833376    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-podinfo" pod "cda46604-6d15-4ee9-823f-cb5cb70f4c51" (UID: "cda46604-6d15-4ee9-823f-cb5cb70f4c51"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:11:40.912198 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.912165    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x55j8\" (UniqueName: \"kubernetes.io/projected/337f5ca4-5214-48b2-a02b-671382ec43b2-kube-api-access-x55j8\") pod \"payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d\" (UID: \"337f5ca4-5214-48b2-a02b-671382ec43b2\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d"
Oct 30 19:11:40.912675 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.912208    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/337f5ca4-5214-48b2-a02b-671382ec43b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d\" (UID: \"337f5ca4-5214-48b2-a02b-671382ec43b2\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d"
Oct 30 19:11:40.912675 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.912355    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:40.912675 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.912365    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:40.912675 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.912374    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:40.912675 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.912383    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cda46604-6d15-4ee9-823f-cb5cb70f4c51-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:40.912675 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.912401    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cda46604-6d15-4ee9-823f-cb5cb70f4c51-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:40.912675 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.912412    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8kk8\" (UniqueName: \"kubernetes.io/projected/cda46604-6d15-4ee9-823f-cb5cb70f4c51-kube-api-access-m8kk8\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:40.913246 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.913173    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/337f5ca4-5214-48b2-a02b-671382ec43b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d\" (UID: \"337f5ca4-5214-48b2-a02b-671382ec43b2\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d"
Oct 30 19:11:40.973959 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:40.973911    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x55j8\" (UniqueName: \"kubernetes.io/projected/337f5ca4-5214-48b2-a02b-671382ec43b2-kube-api-access-x55j8\") pod \"payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d\" (UID: \"337f5ca4-5214-48b2-a02b-671382ec43b2\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d"
Oct 30 19:11:41.058973 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:41.058300    5820 patch_prober.go:29] interesting pod/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 19:11:41.058973 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:41.058363    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6" podUID="4a0996ab-5aaa-4b71-b064-87d9d1572d13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 19:11:41.091848 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:41.091815    5820 generic.go:356] "Generic (PLEG): container finished" podID="cda46604-6d15-4ee9-823f-cb5cb70f4c51" containerID="b437d90c6fa5ae131fb8ae9af8bf2ca377a3d06e4799dac75ee77ba890e48e82" exitCode=0
Oct 30 19:11:41.092040 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:41.091864    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98" event={"ID":"cda46604-6d15-4ee9-823f-cb5cb70f4c51","Type":"ContainerDied","Data":"b437d90c6fa5ae131fb8ae9af8bf2ca377a3d06e4799dac75ee77ba890e48e82"}
Oct 30 19:11:41.092040 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:41.091876    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"
Oct 30 19:11:41.092040 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:41.091967    5820 scope.go:122] "RemoveContainer" containerID="b437d90c6fa5ae131fb8ae9af8bf2ca377a3d06e4799dac75ee77ba890e48e82"
Oct 30 19:11:41.092040 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:41.091941    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98" event={"ID":"cda46604-6d15-4ee9-823f-cb5cb70f4c51","Type":"ContainerDied","Data":"c52849627446ff1fd87f8479c13a2f510988415ef5ab3d12f30635ef1d6c0672"}
Oct 30 19:11:41.102821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:41.102802    5820 scope.go:122] "RemoveContainer" containerID="488e8b395f1d7baa8912b4bda18e4ba773af44820105b23c6943603e130aa9df"
Oct 30 19:11:41.113095 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:41.113074    5820 scope.go:122] "RemoveContainer" containerID="b437d90c6fa5ae131fb8ae9af8bf2ca377a3d06e4799dac75ee77ba890e48e82"
Oct 30 19:11:41.113359 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:11:41.113338    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b437d90c6fa5ae131fb8ae9af8bf2ca377a3d06e4799dac75ee77ba890e48e82\": container with ID starting with b437d90c6fa5ae131fb8ae9af8bf2ca377a3d06e4799dac75ee77ba890e48e82 not found: ID does not exist" containerID="b437d90c6fa5ae131fb8ae9af8bf2ca377a3d06e4799dac75ee77ba890e48e82"
Oct 30 19:11:41.113413 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:41.113369    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b437d90c6fa5ae131fb8ae9af8bf2ca377a3d06e4799dac75ee77ba890e48e82"} err="failed to get container status \"b437d90c6fa5ae131fb8ae9af8bf2ca377a3d06e4799dac75ee77ba890e48e82\": rpc error: code = NotFound desc = could not find container \"b437d90c6fa5ae131fb8ae9af8bf2ca377a3d06e4799dac75ee77ba890e48e82\": container with ID starting with b437d90c6fa5ae131fb8ae9af8bf2ca377a3d06e4799dac75ee77ba890e48e82 not found: ID does not exist"
Oct 30 19:11:41.113413 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:41.113386    5820 scope.go:122] "RemoveContainer" containerID="488e8b395f1d7baa8912b4bda18e4ba773af44820105b23c6943603e130aa9df"
Oct 30 19:11:41.113621 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:11:41.113606    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"488e8b395f1d7baa8912b4bda18e4ba773af44820105b23c6943603e130aa9df\": container with ID starting with 488e8b395f1d7baa8912b4bda18e4ba773af44820105b23c6943603e130aa9df not found: ID does not exist" containerID="488e8b395f1d7baa8912b4bda18e4ba773af44820105b23c6943603e130aa9df"
Oct 30 19:11:41.113665 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:41.113626    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"488e8b395f1d7baa8912b4bda18e4ba773af44820105b23c6943603e130aa9df"} err="failed to get container status \"488e8b395f1d7baa8912b4bda18e4ba773af44820105b23c6943603e130aa9df\": rpc error: code = NotFound desc = could not find container \"488e8b395f1d7baa8912b4bda18e4ba773af44820105b23c6943603e130aa9df\": container with ID starting with 488e8b395f1d7baa8912b4bda18e4ba773af44820105b23c6943603e130aa9df not found: ID does not exist"
Oct 30 19:11:41.185834 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:41.185809    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d"
Oct 30 19:11:41.275566 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:41.275250    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"]
Oct 30 19:11:41.278554 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:41.278529    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-e67448-openshift-default-568d47b899-v6w98"]
Oct 30 19:11:41.461923 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:41.461837    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cda46604-6d15-4ee9-823f-cb5cb70f4c51" path="/var/lib/kubelet/pods/cda46604-6d15-4ee9-823f-cb5cb70f4c51/volumes"
Oct 30 19:11:41.581290 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:41.581251    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d"]
Oct 30 19:11:41.582974 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:11:41.582927    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod337f5ca4_5214_48b2_a02b_671382ec43b2.slice/crio-bdb9bdb57d1eb429cadab5f9e0521c9442242804a08f3558f0efaf95bb66dae7 WatchSource:0}: Error finding container bdb9bdb57d1eb429cadab5f9e0521c9442242804a08f3558f0efaf95bb66dae7: Status 404 returned error can't find the container with id bdb9bdb57d1eb429cadab5f9e0521c9442242804a08f3558f0efaf95bb66dae7
Oct 30 19:11:42.033977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:42.033920    5820 patch_prober.go:29] interesting pod/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 19:11:42.034489 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:42.034038    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6" podUID="4a0996ab-5aaa-4b71-b064-87d9d1572d13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 19:11:42.099418 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:42.099380    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d" event={"ID":"337f5ca4-5214-48b2-a02b-671382ec43b2","Type":"ContainerStarted","Data":"6138b54e21a1f4a2fb4e88b6788994e706a8ba1785351d2f1e5964f8aae86820"}
Oct 30 19:11:42.099623 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:42.099433    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d" event={"ID":"337f5ca4-5214-48b2-a02b-671382ec43b2","Type":"ContainerStarted","Data":"bdb9bdb57d1eb429cadab5f9e0521c9442242804a08f3558f0efaf95bb66dae7"}
Oct 30 19:11:42.099623 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:42.099474    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d"
Oct 30 19:11:42.108243 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:42.108217    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:42.222281 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:42.222229    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d" podStartSLOduration=2.222214918 podStartE2EDuration="2.222214918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:11:42.220205289 +0000 UTC m=+2222.931953987" watchObservedRunningTime="2026-10-30 19:11:42.222214918 +0000 UTC m=+2222.933963614"
Oct 30 19:11:42.362980 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:42.355926    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:43.036982 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:43.034243    5820 patch_prober.go:29] interesting pod/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 19:11:43.036982 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:43.034301    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6" podUID="4a0996ab-5aaa-4b71-b064-87d9d1572d13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 19:11:44.038857 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:44.038829    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:11:44.039345 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:44.039293    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:11:44.039629 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:44.039611    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:11:45.410129 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:45.410080    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:45.636334 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:45.636297    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:45.806301 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:45.806267    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72"]
Oct 30 19:11:45.806943 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:45.806919    5820 cpu_manager.go:414] "containerMap: removing container" podUID="cda46604-6d15-4ee9-823f-cb5cb70f4c51" containerName="istio-proxy"
Oct 30 19:11:45.806943 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:45.806940    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cda46604-6d15-4ee9-823f-cb5cb70f4c51" containerName="istio-proxy"
Oct 30 19:11:45.807188 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:45.806995    5820 cpu_manager.go:414] "containerMap: removing container" podUID="cda46604-6d15-4ee9-823f-cb5cb70f4c51" containerName="istio-proxy"
Oct 30 19:11:45.807188 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:45.807001    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cda46604-6d15-4ee9-823f-cb5cb70f4c51" containerName="istio-proxy"
Oct 30 19:11:45.807188 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:45.807115    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="cda46604-6d15-4ee9-823f-cb5cb70f4c51" containerName="istio-proxy"
Oct 30 19:11:45.807188 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:45.807127    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="cda46604-6d15-4ee9-823f-cb5cb70f4c51" containerName="istio-proxy"
Oct 30 19:11:45.809443 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:45.809419    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72"
Oct 30 19:11:45.913768 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:45.913736    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-52d383\""
Oct 30 19:11:45.914204 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:45.914170    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-52d383-dockercfg-hfnrn\""
Oct 30 19:11:45.920802 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:45.920776    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72"]
Oct 30 19:11:45.958056 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:45.958017    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c85bce08-7b30-4c2f-bc48-36f60dfad8d2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72\" (UID: \"c85bce08-7b30-4c2f-bc48-36f60dfad8d2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72"
Oct 30 19:11:45.985798 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:45.985760    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnttn\" (UniqueName: \"kubernetes.io/projected/c85bce08-7b30-4c2f-bc48-36f60dfad8d2-kube-api-access-dnttn\") pod \"payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72\" (UID: \"c85bce08-7b30-4c2f-bc48-36f60dfad8d2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72"
Oct 30 19:11:45.997099 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:45.997068    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h"]
Oct 30 19:11:46.004486 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:46.004228    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h"
Oct 30 19:11:46.011320 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:46.011265    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h"]
Oct 30 19:11:46.087676 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:46.087602    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnxfl\" (UniqueName: \"kubernetes.io/projected/4923d0a0-12b1-4d37-b17c-bc7369ba6dad-kube-api-access-qnxfl\") pod \"payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h\" (UID: \"4923d0a0-12b1-4d37-b17c-bc7369ba6dad\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h"
Oct 30 19:11:46.087818 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:46.087702    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c85bce08-7b30-4c2f-bc48-36f60dfad8d2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72\" (UID: \"c85bce08-7b30-4c2f-bc48-36f60dfad8d2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72"
Oct 30 19:11:46.087818 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:46.087761    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4923d0a0-12b1-4d37-b17c-bc7369ba6dad-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h\" (UID: \"4923d0a0-12b1-4d37-b17c-bc7369ba6dad\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h"
Oct 30 19:11:46.087934 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:46.087908    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnttn\" (UniqueName: \"kubernetes.io/projected/c85bce08-7b30-4c2f-bc48-36f60dfad8d2-kube-api-access-dnttn\") pod \"payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72\" (UID: \"c85bce08-7b30-4c2f-bc48-36f60dfad8d2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72"
Oct 30 19:11:46.088509 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:46.088491    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c85bce08-7b30-4c2f-bc48-36f60dfad8d2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72\" (UID: \"c85bce08-7b30-4c2f-bc48-36f60dfad8d2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72"
Oct 30 19:11:46.119786 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:46.119756    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:46.161924 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:46.161887    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnttn\" (UniqueName: \"kubernetes.io/projected/c85bce08-7b30-4c2f-bc48-36f60dfad8d2-kube-api-access-dnttn\") pod \"payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72\" (UID: \"c85bce08-7b30-4c2f-bc48-36f60dfad8d2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72"
Oct 30 19:11:46.189293 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:46.189258    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnxfl\" (UniqueName: \"kubernetes.io/projected/4923d0a0-12b1-4d37-b17c-bc7369ba6dad-kube-api-access-qnxfl\") pod \"payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h\" (UID: \"4923d0a0-12b1-4d37-b17c-bc7369ba6dad\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h"
Oct 30 19:11:46.189419 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:46.189323    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4923d0a0-12b1-4d37-b17c-bc7369ba6dad-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h\" (UID: \"4923d0a0-12b1-4d37-b17c-bc7369ba6dad\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h"
Oct 30 19:11:46.190101 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:46.190081    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4923d0a0-12b1-4d37-b17c-bc7369ba6dad-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h\" (UID: \"4923d0a0-12b1-4d37-b17c-bc7369ba6dad\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h"
Oct 30 19:11:46.250944 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:46.250915    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnxfl\" (UniqueName: \"kubernetes.io/projected/4923d0a0-12b1-4d37-b17c-bc7369ba6dad-kube-api-access-qnxfl\") pod \"payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h\" (UID: \"4923d0a0-12b1-4d37-b17c-bc7369ba6dad\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h"
Oct 30 19:11:46.323045 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:46.323013    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h"
Oct 30 19:11:46.422376 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:46.422344    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72"
Oct 30 19:11:46.700869 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:46.700836    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h"]
Oct 30 19:11:46.702027 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:11:46.701998    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4923d0a0_12b1_4d37_b17c_bc7369ba6dad.slice/crio-b5753f942bfa7dca9f086c325035c56b20fe5b4619fd9a41e045314083be41b4 WatchSource:0}: Error finding container b5753f942bfa7dca9f086c325035c56b20fe5b4619fd9a41e045314083be41b4: Status 404 returned error can't find the container with id b5753f942bfa7dca9f086c325035c56b20fe5b4619fd9a41e045314083be41b4
Oct 30 19:11:46.800941 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:46.800917    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72"]
Oct 30 19:11:46.802404 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:11:46.802378    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc85bce08_7b30_4c2f_bc48_36f60dfad8d2.slice/crio-f30927d1e4bae7713c8954a12a426a638cc27bcc9d30e5026f45d8ba94309846 WatchSource:0}: Error finding container f30927d1e4bae7713c8954a12a426a638cc27bcc9d30e5026f45d8ba94309846: Status 404 returned error can't find the container with id f30927d1e4bae7713c8954a12a426a638cc27bcc9d30e5026f45d8ba94309846
Oct 30 19:11:47.140563 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:47.140520    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h" event={"ID":"4923d0a0-12b1-4d37-b17c-bc7369ba6dad","Type":"ContainerStarted","Data":"203b782717308cf22250645ad4e023a4afecad9d2b325431bcd9777e7c911101"}
Oct 30 19:11:47.140823 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:47.140585    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h" event={"ID":"4923d0a0-12b1-4d37-b17c-bc7369ba6dad","Type":"ContainerStarted","Data":"b5753f942bfa7dca9f086c325035c56b20fe5b4619fd9a41e045314083be41b4"}
Oct 30 19:11:47.140823 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:47.140647    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h"
Oct 30 19:11:47.142645 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:47.142619    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72" event={"ID":"c85bce08-7b30-4c2f-bc48-36f60dfad8d2","Type":"ContainerStarted","Data":"a3289f317c7f8595a5569e0210a1fc9f100ea2c994cda57a6a7878ed6017b1db"}
Oct 30 19:11:47.142747 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:47.142664    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72" event={"ID":"c85bce08-7b30-4c2f-bc48-36f60dfad8d2","Type":"ContainerStarted","Data":"f30927d1e4bae7713c8954a12a426a638cc27bcc9d30e5026f45d8ba94309846"}
Oct 30 19:11:47.143177 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:47.143157    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72"
Oct 30 19:11:47.261123 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:47.261078    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h" podStartSLOduration=2.261066479 podStartE2EDuration="2.261066479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:11:47.258167424 +0000 UTC m=+2227.969916120" watchObservedRunningTime="2026-10-30 19:11:47.261066479 +0000 UTC m=+2227.972815175"
Oct 30 19:11:47.391213 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:47.388092    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72" podStartSLOduration=2.388068238 podStartE2EDuration="2.388068238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:11:47.375320556 +0000 UTC m=+2228.087069262" watchObservedRunningTime="2026-10-30 19:11:47.388068238 +0000 UTC m=+2228.099816936"
Oct 30 19:11:49.748642 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:49.748608    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:50.238495 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:50.238454    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:52.850981 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:52.850924    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:53.114887 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:53.114787    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d"
Oct 30 19:11:57.790222 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:57.790182    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:58.153332 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:58.153305    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h"
Oct 30 19:11:59.017981 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:59.017922    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:59.159374 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:11:59.159342    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72"
Oct 30 19:12:01.950820 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:01.950776    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:02.928279 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:02.914097    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:03.184716 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:03.184619    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:03.688809 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:03.688779    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"]
Oct 30 19:12:03.693468 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:03.693445    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:03.763976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:03.752300    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-cf43e7-openshift-default-dockercfg-t7cwd\""
Oct 30 19:12:03.893395 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:03.893325    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"]
Oct 30 19:12:03.900667 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:03.900632    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-workload-socket\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:03.903081 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:03.902873    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:03.903365 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:03.903346    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd708157-c511-4334-84c0-6d0a655e3502-istio-token\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:03.903880 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:03.903784    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6dpw\" (UniqueName: \"kubernetes.io/projected/fd708157-c511-4334-84c0-6d0a655e3502-kube-api-access-z6dpw\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:03.904112 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:03.904086    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-istio-envoy\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:03.904577 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:03.904542    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-credential-socket\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:03.905111 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:03.905083    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd708157-c511-4334-84c0-6d0a655e3502-istio-podinfo\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:03.905280 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:03.905251    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-istio-data\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:03.905693 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:03.905581    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-workload-certs\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:03.906248 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:03.905942    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd708157-c511-4334-84c0-6d0a655e3502-istiod-ca-cert\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:04.008687 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.008349    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6dpw\" (UniqueName: \"kubernetes.io/projected/fd708157-c511-4334-84c0-6d0a655e3502-kube-api-access-z6dpw\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:04.008687 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.008428    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-istio-envoy\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:04.008687 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.008481    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-credential-socket\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:04.008687 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.008545    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd708157-c511-4334-84c0-6d0a655e3502-istio-podinfo\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:04.008687 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.008570    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-istio-data\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:04.008687 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.008648    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-workload-certs\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:04.008687 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.008677    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd708157-c511-4334-84c0-6d0a655e3502-istiod-ca-cert\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:04.009261 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.008935    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-workload-socket\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:04.009261 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.009023    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd708157-c511-4334-84c0-6d0a655e3502-istio-token\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:04.009261 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.009161    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-workload-certs\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:04.009261 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.009198    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-istio-data\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:04.009489 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.009276    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-credential-socket\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:04.009690 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.009670    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd708157-c511-4334-84c0-6d0a655e3502-istiod-ca-cert\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:04.009788 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.009767    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-workload-socket\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:04.011306 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.011287    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-istio-envoy\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:04.011720 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.011704    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd708157-c511-4334-84c0-6d0a655e3502-istio-podinfo\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:04.076888 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.076850    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6dpw\" (UniqueName: \"kubernetes.io/projected/fd708157-c511-4334-84c0-6d0a655e3502-kube-api-access-z6dpw\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:04.077560 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.077530    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd708157-c511-4334-84c0-6d0a655e3502-istio-token\") pod \"e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") " pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:04.113826 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.113786    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:04.331447 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.331412    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:04.781051 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:12:04.781011    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd708157_c511_4334_84c0_6d0a655e3502.slice/crio-1d7862b3b7c9a567c8cec72df6b7f32aed823d3e5d0ec99a162dd9e85811ee6d WatchSource:0}: Error finding container 1d7862b3b7c9a567c8cec72df6b7f32aed823d3e5d0ec99a162dd9e85811ee6d: Status 404 returned error can't find the container with id 1d7862b3b7c9a567c8cec72df6b7f32aed823d3e5d0ec99a162dd9e85811ee6d
Oct 30 19:12:04.781646 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.781616    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"]
Oct 30 19:12:04.783624 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.783592    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:12:04.783716 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.783656    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:12:04.783716 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:04.783686    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:12:05.258028 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:05.257981    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq" event={"ID":"fd708157-c511-4334-84c0-6d0a655e3502","Type":"ContainerStarted","Data":"6c51e3a8f0375ca46d81242e388b2c87378f348315273f154900a53e407d64cc"}
Oct 30 19:12:05.258241 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:05.258066    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq" event={"ID":"fd708157-c511-4334-84c0-6d0a655e3502","Type":"ContainerStarted","Data":"1d7862b3b7c9a567c8cec72df6b7f32aed823d3e5d0ec99a162dd9e85811ee6d"}
Oct 30 19:12:05.332700 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:05.332674    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:05.338329 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:05.338306    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:05.383474 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:05.383419    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq" podStartSLOduration=2.383405447 podStartE2EDuration="2.383405447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:05.379030029 +0000 UTC m=+2246.090778723" watchObservedRunningTime="2026-10-30 19:12:05.383405447 +0000 UTC m=+2246.095154143"
Oct 30 19:12:06.263906 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:06.263875    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:06.264713 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:06.264695    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:12:07.884504 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:07.884469    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:08.204922 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:12:08.204840    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:12:08.626702 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:08.626668    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:08.802565 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:08.802529    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:08.993573 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:08.993486    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:09.179914 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:09.179879    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:13.297981 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:13.296279    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:13.590402 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:13.590317    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:13.765098 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:13.765061    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:13.970341 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:13.970266    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8"]
Oct 30 19:12:13.975490 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:13.975474    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8"
Oct 30 19:12:14.032492 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:14.032464    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8"]
Oct 30 19:12:14.065635 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:14.065470    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47db0d9f-9872-494d-aa71-f541802898f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8\" (UID: \"47db0d9f-9872-494d-aa71-f541802898f7\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8"
Oct 30 19:12:14.065635 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:14.065576    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66d8g\" (UniqueName: \"kubernetes.io/projected/47db0d9f-9872-494d-aa71-f541802898f7-kube-api-access-66d8g\") pod \"payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8\" (UID: \"47db0d9f-9872-494d-aa71-f541802898f7\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8"
Oct 30 19:12:14.103029 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:14.096300    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-b-cf43e7\""
Oct 30 19:12:14.103029 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:14.096468    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-b-cf43e7-dockercfg-8p964\""
Oct 30 19:12:14.181973 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:14.168964    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l"]
Oct 30 19:12:14.181973 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:14.177233    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l"
Oct 30 19:12:14.181973 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:14.181446    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l"]
Oct 30 19:12:14.183311 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:14.183283    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47db0d9f-9872-494d-aa71-f541802898f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8\" (UID: \"47db0d9f-9872-494d-aa71-f541802898f7\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8"
Oct 30 19:12:14.183579 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:14.183557    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66d8g\" (UniqueName: \"kubernetes.io/projected/47db0d9f-9872-494d-aa71-f541802898f7-kube-api-access-66d8g\") pod \"payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8\" (UID: \"47db0d9f-9872-494d-aa71-f541802898f7\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8"
Oct 30 19:12:14.184997 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:14.184978    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47db0d9f-9872-494d-aa71-f541802898f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8\" (UID: \"47db0d9f-9872-494d-aa71-f541802898f7\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8"
Oct 30 19:12:14.247916 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:14.247849    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66d8g\" (UniqueName: \"kubernetes.io/projected/47db0d9f-9872-494d-aa71-f541802898f7-kube-api-access-66d8g\") pod \"payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8\" (UID: \"47db0d9f-9872-494d-aa71-f541802898f7\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8"
Oct 30 19:12:14.284978 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:14.284913    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6da26a61-a369-4f91-ad4d-d4b8510b6055-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l\" (UID: \"6da26a61-a369-4f91-ad4d-d4b8510b6055\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l"
Oct 30 19:12:14.285144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:14.285130    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xw8g\" (UniqueName: \"kubernetes.io/projected/6da26a61-a369-4f91-ad4d-d4b8510b6055-kube-api-access-5xw8g\") pod \"payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l\" (UID: \"6da26a61-a369-4f91-ad4d-d4b8510b6055\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l"
Oct 30 19:12:14.290585 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:14.290564    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8"
Oct 30 19:12:14.386559 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:14.386514    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6da26a61-a369-4f91-ad4d-d4b8510b6055-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l\" (UID: \"6da26a61-a369-4f91-ad4d-d4b8510b6055\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l"
Oct 30 19:12:14.387051 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:14.386742    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xw8g\" (UniqueName: \"kubernetes.io/projected/6da26a61-a369-4f91-ad4d-d4b8510b6055-kube-api-access-5xw8g\") pod \"payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l\" (UID: \"6da26a61-a369-4f91-ad4d-d4b8510b6055\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l"
Oct 30 19:12:14.387431 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:14.387405    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6da26a61-a369-4f91-ad4d-d4b8510b6055-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l\" (UID: \"6da26a61-a369-4f91-ad4d-d4b8510b6055\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l"
Oct 30 19:12:14.448522 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:14.448492    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xw8g\" (UniqueName: \"kubernetes.io/projected/6da26a61-a369-4f91-ad4d-d4b8510b6055-kube-api-access-5xw8g\") pod \"payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l\" (UID: \"6da26a61-a369-4f91-ad4d-d4b8510b6055\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l"
Oct 30 19:12:14.504412 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:14.504340    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l"
Oct 30 19:12:14.679259 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:14.679228    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8"]
Oct 30 19:12:14.679981 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:12:14.679928    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47db0d9f_9872_494d_aa71_f541802898f7.slice/crio-ea199202bbbdeb6304ebeb67dd82bf475ae86d848134da9c60f84274331bf111 WatchSource:0}: Error finding container ea199202bbbdeb6304ebeb67dd82bf475ae86d848134da9c60f84274331bf111: Status 404 returned error can't find the container with id ea199202bbbdeb6304ebeb67dd82bf475ae86d848134da9c60f84274331bf111
Oct 30 19:12:14.885678 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:14.885648    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l"]
Oct 30 19:12:14.887190 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:12:14.887166    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6da26a61_a369_4f91_ad4d_d4b8510b6055.slice/crio-d080ca4851626045d99bcc1bd6a9ba552d583156bf1005887c42f63ba051c4d5 WatchSource:0}: Error finding container d080ca4851626045d99bcc1bd6a9ba552d583156bf1005887c42f63ba051c4d5: Status 404 returned error can't find the container with id d080ca4851626045d99bcc1bd6a9ba552d583156bf1005887c42f63ba051c4d5
Oct 30 19:12:15.330876 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:15.330842    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8" event={"ID":"47db0d9f-9872-494d-aa71-f541802898f7","Type":"ContainerStarted","Data":"a94c20b1dbc693c12c2aa306c76ffcf8a9adc7796780033afa182b0f3169b0b3"}
Oct 30 19:12:15.331114 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:15.330901    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8"
Oct 30 19:12:15.331114 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:15.330920    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8" event={"ID":"47db0d9f-9872-494d-aa71-f541802898f7","Type":"ContainerStarted","Data":"ea199202bbbdeb6304ebeb67dd82bf475ae86d848134da9c60f84274331bf111"}
Oct 30 19:12:15.332522 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:15.332494    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l" event={"ID":"6da26a61-a369-4f91-ad4d-d4b8510b6055","Type":"ContainerStarted","Data":"f90f9fdce9cd7fe9898f09fa3331f02699c8833dc3988ab5fb2e7fee0a360da6"}
Oct 30 19:12:15.332649 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:15.332542    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l" event={"ID":"6da26a61-a369-4f91-ad4d-d4b8510b6055","Type":"ContainerStarted","Data":"d080ca4851626045d99bcc1bd6a9ba552d583156bf1005887c42f63ba051c4d5"}
Oct 30 19:12:15.332649 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:15.332567    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l"
Oct 30 19:12:15.449605 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:15.449551    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8" podStartSLOduration=2.44953366 podStartE2EDuration="2.44953366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:15.447058635 +0000 UTC m=+2256.158807343" watchObservedRunningTime="2026-10-30 19:12:15.44953366 +0000 UTC m=+2256.161282358"
Oct 30 19:12:15.591423 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:15.578638    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l" podStartSLOduration=1.578619078 podStartE2EDuration="1.578619078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:15.575362643 +0000 UTC m=+2256.287111342" watchObservedRunningTime="2026-10-30 19:12:15.578619078 +0000 UTC m=+2256.290367778"
Oct 30 19:12:18.368633 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:18.368544    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:23.179975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:23.179848    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:25.444810 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:25.444773    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx"]
Oct 30 19:12:25.554463 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:25.554430    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8"]
Oct 30 19:12:25.891699 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:25.882450    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:26.075884 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:26.075845    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:26.342064 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:26.342039    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l"
Oct 30 19:12:26.342269 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:26.342116    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8"
Oct 30 19:12:28.112455 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:28.112392    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:30.445238 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:30.445202    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx" podUID="54b02e73-6065-40b6-b0cb-bc2136f9b83e" containerName="payload-processing" containerID="cri-o://fc2b4fa99e76fdc395e1396c95507ce114e14e64e3626366a76bd04cce41568d" gracePeriod=25
Oct 30 19:12:30.555639 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:30.555591    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8" podUID="3128fea4-f055-4fde-99e5-e71a21abe185" containerName="payload-pre-processing" containerID="cri-o://aa9ac7f8d2557555139624e78f9ac6f938f76d324f03045c4e9960a9cc803599" gracePeriod=25
Oct 30 19:12:30.709283 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:30.709216    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx"
Oct 30 19:12:30.787052 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:30.787017    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/54b02e73-6065-40b6-b0cb-bc2136f9b83e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54b02e73-6065-40b6-b0cb-bc2136f9b83e-plugins-config-volume\") pod \"54b02e73-6065-40b6-b0cb-bc2136f9b83e\" (UID: \"54b02e73-6065-40b6-b0cb-bc2136f9b83e\") "
Oct 30 19:12:30.787220 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:30.787143    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54b02e73-6065-40b6-b0cb-bc2136f9b83e-kube-api-access-f94tx\" (UniqueName: \"kubernetes.io/projected/54b02e73-6065-40b6-b0cb-bc2136f9b83e-kube-api-access-f94tx\") pod \"54b02e73-6065-40b6-b0cb-bc2136f9b83e\" (UID: \"54b02e73-6065-40b6-b0cb-bc2136f9b83e\") "
Oct 30 19:12:30.787505 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:30.787478    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54b02e73-6065-40b6-b0cb-bc2136f9b83e-plugins-config-volume" pod "54b02e73-6065-40b6-b0cb-bc2136f9b83e" (UID: "54b02e73-6065-40b6-b0cb-bc2136f9b83e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:12:30.787724 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:30.787705    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54b02e73-6065-40b6-b0cb-bc2136f9b83e-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:12:30.790787 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:30.790752    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54b02e73-6065-40b6-b0cb-bc2136f9b83e-kube-api-access-f94tx" pod "54b02e73-6065-40b6-b0cb-bc2136f9b83e" (UID: "54b02e73-6065-40b6-b0cb-bc2136f9b83e"). InnerVolumeSpecName "kube-api-access-f94tx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:12:30.828111 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:30.828086    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8"
Oct 30 19:12:30.888902 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:30.888875    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3128fea4-f055-4fde-99e5-e71a21abe185-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3128fea4-f055-4fde-99e5-e71a21abe185-plugins-config-volume\") pod \"3128fea4-f055-4fde-99e5-e71a21abe185\" (UID: \"3128fea4-f055-4fde-99e5-e71a21abe185\") "
Oct 30 19:12:30.889116 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:30.889040    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3128fea4-f055-4fde-99e5-e71a21abe185-kube-api-access-gbfhz\" (UniqueName: \"kubernetes.io/projected/3128fea4-f055-4fde-99e5-e71a21abe185-kube-api-access-gbfhz\") pod \"3128fea4-f055-4fde-99e5-e71a21abe185\" (UID: \"3128fea4-f055-4fde-99e5-e71a21abe185\") "
Oct 30 19:12:30.889307 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:30.889283    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3128fea4-f055-4fde-99e5-e71a21abe185-plugins-config-volume" pod "3128fea4-f055-4fde-99e5-e71a21abe185" (UID: "3128fea4-f055-4fde-99e5-e71a21abe185"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:12:30.889550 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:30.889533    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f94tx\" (UniqueName: \"kubernetes.io/projected/54b02e73-6065-40b6-b0cb-bc2136f9b83e-kube-api-access-f94tx\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:12:30.889615 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:30.889558    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3128fea4-f055-4fde-99e5-e71a21abe185-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:12:30.891255 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:30.891236    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3128fea4-f055-4fde-99e5-e71a21abe185-kube-api-access-gbfhz" pod "3128fea4-f055-4fde-99e5-e71a21abe185" (UID: "3128fea4-f055-4fde-99e5-e71a21abe185"). InnerVolumeSpecName "kube-api-access-gbfhz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:12:30.990599 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:30.990529    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbfhz\" (UniqueName: \"kubernetes.io/projected/3128fea4-f055-4fde-99e5-e71a21abe185-kube-api-access-gbfhz\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:12:31.437479 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:31.437436    5820 generic.go:356] "Generic (PLEG): container finished" podID="3128fea4-f055-4fde-99e5-e71a21abe185" containerID="aa9ac7f8d2557555139624e78f9ac6f938f76d324f03045c4e9960a9cc803599" exitCode=0
Oct 30 19:12:31.437688 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:31.437535    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8"
Oct 30 19:12:31.438219 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:31.438191    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8" event={"ID":"3128fea4-f055-4fde-99e5-e71a21abe185","Type":"ContainerDied","Data":"aa9ac7f8d2557555139624e78f9ac6f938f76d324f03045c4e9960a9cc803599"}
Oct 30 19:12:31.438352 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:31.438243    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8" event={"ID":"3128fea4-f055-4fde-99e5-e71a21abe185","Type":"ContainerDied","Data":"a989b29f0a796b8eb2f46d2252ebadd8ff59ff2343587f93d96a286f4d0c8a8e"}
Oct 30 19:12:31.438352 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:31.438276    5820 scope.go:122] "RemoveContainer" containerID="aa9ac7f8d2557555139624e78f9ac6f938f76d324f03045c4e9960a9cc803599"
Oct 30 19:12:31.439141 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:31.439112    5820 generic.go:356] "Generic (PLEG): container finished" podID="54b02e73-6065-40b6-b0cb-bc2136f9b83e" containerID="fc2b4fa99e76fdc395e1396c95507ce114e14e64e3626366a76bd04cce41568d" exitCode=0
Oct 30 19:12:31.439253 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:31.439150    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx"
Oct 30 19:12:31.439253 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:31.439201    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx" event={"ID":"54b02e73-6065-40b6-b0cb-bc2136f9b83e","Type":"ContainerDied","Data":"fc2b4fa99e76fdc395e1396c95507ce114e14e64e3626366a76bd04cce41568d"}
Oct 30 19:12:31.439384 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:31.439247    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx" event={"ID":"54b02e73-6065-40b6-b0cb-bc2136f9b83e","Type":"ContainerDied","Data":"03bf8ae8d68f2e9eecee20393da0f7aee37e47efa072c4dd871dd830ef3940ec"}
Oct 30 19:12:31.458536 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:31.458435    5820 scope.go:122] "RemoveContainer" containerID="aa9ac7f8d2557555139624e78f9ac6f938f76d324f03045c4e9960a9cc803599"
Oct 30 19:12:31.458796 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:12:31.458754    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa9ac7f8d2557555139624e78f9ac6f938f76d324f03045c4e9960a9cc803599\": container with ID starting with aa9ac7f8d2557555139624e78f9ac6f938f76d324f03045c4e9960a9cc803599 not found: ID does not exist" containerID="aa9ac7f8d2557555139624e78f9ac6f938f76d324f03045c4e9960a9cc803599"
Oct 30 19:12:31.458796 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:31.458781    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa9ac7f8d2557555139624e78f9ac6f938f76d324f03045c4e9960a9cc803599"} err="failed to get container status \"aa9ac7f8d2557555139624e78f9ac6f938f76d324f03045c4e9960a9cc803599\": rpc error: code = NotFound desc = could not find container \"aa9ac7f8d2557555139624e78f9ac6f938f76d324f03045c4e9960a9cc803599\": container with ID starting with aa9ac7f8d2557555139624e78f9ac6f938f76d324f03045c4e9960a9cc803599 not found: ID does not exist"
Oct 30 19:12:31.458879 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:31.458800    5820 scope.go:122] "RemoveContainer" containerID="fc2b4fa99e76fdc395e1396c95507ce114e14e64e3626366a76bd04cce41568d"
Oct 30 19:12:31.474702 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:31.474679    5820 scope.go:122] "RemoveContainer" containerID="fc2b4fa99e76fdc395e1396c95507ce114e14e64e3626366a76bd04cce41568d"
Oct 30 19:12:31.475020 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:12:31.474993    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc2b4fa99e76fdc395e1396c95507ce114e14e64e3626366a76bd04cce41568d\": container with ID starting with fc2b4fa99e76fdc395e1396c95507ce114e14e64e3626366a76bd04cce41568d not found: ID does not exist" containerID="fc2b4fa99e76fdc395e1396c95507ce114e14e64e3626366a76bd04cce41568d"
Oct 30 19:12:31.475104 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:31.475029    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc2b4fa99e76fdc395e1396c95507ce114e14e64e3626366a76bd04cce41568d"} err="failed to get container status \"fc2b4fa99e76fdc395e1396c95507ce114e14e64e3626366a76bd04cce41568d\": rpc error: code = NotFound desc = could not find container \"fc2b4fa99e76fdc395e1396c95507ce114e14e64e3626366a76bd04cce41568d\": container with ID starting with fc2b4fa99e76fdc395e1396c95507ce114e14e64e3626366a76bd04cce41568d not found: ID does not exist"
Oct 30 19:12:31.603392 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:31.603361    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8"]
Oct 30 19:12:31.653125 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:31.653093    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-83ab2c-7499f8559-lcwd8"]
Oct 30 19:12:31.718669 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:31.718604    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx"]
Oct 30 19:12:31.725294 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:31.725272    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-83ab2c-96c774698-r87dx"]
Oct 30 19:12:33.437699 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:33.437666    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3128fea4-f055-4fde-99e5-e71a21abe185" path="/var/lib/kubelet/pods/3128fea4-f055-4fde-99e5-e71a21abe185/volumes"
Oct 30 19:12:33.438494 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:33.438106    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="54b02e73-6065-40b6-b0cb-bc2136f9b83e" path="/var/lib/kubelet/pods/54b02e73-6065-40b6-b0cb-bc2136f9b83e/volumes"
Oct 30 19:12:34.366548 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.366514    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"]
Oct 30 19:12:34.367180 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.367166    5820 cpu_manager.go:414] "containerMap: removing container" podUID="54b02e73-6065-40b6-b0cb-bc2136f9b83e" containerName="payload-processing"
Oct 30 19:12:34.367234 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.367181    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54b02e73-6065-40b6-b0cb-bc2136f9b83e" containerName="payload-processing"
Oct 30 19:12:34.367234 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.367203    5820 cpu_manager.go:414] "containerMap: removing container" podUID="3128fea4-f055-4fde-99e5-e71a21abe185" containerName="payload-pre-processing"
Oct 30 19:12:34.367234 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.367209    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3128fea4-f055-4fde-99e5-e71a21abe185" containerName="payload-pre-processing"
Oct 30 19:12:34.367340 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.367303    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="54b02e73-6065-40b6-b0cb-bc2136f9b83e" containerName="payload-processing"
Oct 30 19:12:34.367340 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.367314    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="3128fea4-f055-4fde-99e5-e71a21abe185" containerName="payload-pre-processing"
Oct 30 19:12:34.372229 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.372206    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"
Oct 30 19:12:34.426732 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.426700    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-f554bb-serving-cert\""
Oct 30 19:12:34.428236 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.428215    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x\" (UID: \"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"
Oct 30 19:12:34.428329 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.428315    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-maas-api-tls\") pod \"maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x\" (UID: \"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"
Oct 30 19:12:34.428418 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.428397    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n64r2\" (UniqueName: \"kubernetes.io/projected/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-kube-api-access-n64r2\") pod \"maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x\" (UID: \"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"
Oct 30 19:12:34.428485 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.428473    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-metrics-certs\") pod \"maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x\" (UID: \"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"
Oct 30 19:12:34.478467 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.478439    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"]
Oct 30 19:12:34.529456 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.529427    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x\" (UID: \"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"
Oct 30 19:12:34.529631 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.529526    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-maas-api-tls\") pod \"maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x\" (UID: \"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"
Oct 30 19:12:34.529631 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.529568    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n64r2\" (UniqueName: \"kubernetes.io/projected/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-kube-api-access-n64r2\") pod \"maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x\" (UID: \"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"
Oct 30 19:12:34.529762 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.529640    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-metrics-certs\") pod \"maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x\" (UID: \"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"
Oct 30 19:12:34.529762 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:12:34.529677    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-serving-cert: secret "maas-api-e2e-praxis-a-f554bb-serving-cert" not found
Oct 30 19:12:34.529762 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:12:34.529756    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-maas-api-tls podName:174e6ae1-f7a9-4ee8-94b9-001dfbb9f387 nodeName:}" failed. No retries permitted until 2026-10-30 19:12:35.029731775 +0000 UTC m=+2275.741480454 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-maas-api-tls") pod "maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x" (UID: "174e6ae1-f7a9-4ee8-94b9-001dfbb9f387") : secret "maas-api-e2e-praxis-a-f554bb-serving-cert" not found
Oct 30 19:12:34.530190 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.530168    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x\" (UID: \"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"
Oct 30 19:12:34.532333 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.532309    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-metrics-certs\") pod \"maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x\" (UID: \"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"
Oct 30 19:12:34.590548 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.590520    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n64r2\" (UniqueName: \"kubernetes.io/projected/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-kube-api-access-n64r2\") pod \"maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x\" (UID: \"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"
Oct 30 19:12:34.601540 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.601504    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:34.807037 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.800043    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:34.989426 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:34.989387    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:35.047896 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:35.047865    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-maas-api-tls\") pod \"maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x\" (UID: \"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"
Oct 30 19:12:35.050350 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:35.050324    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-maas-api-tls\") pod \"maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x\" (UID: \"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"
Oct 30 19:12:35.286588 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:35.286512    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"
Oct 30 19:12:35.675922 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:35.675896    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"]
Oct 30 19:12:35.678097 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:12:35.678064    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod174e6ae1_f7a9_4ee8_94b9_001dfbb9f387.slice/crio-f3bf1748577cf75244f62c7653ea9699311901dadb48816215419754f1073202 WatchSource:0}: Error finding container f3bf1748577cf75244f62c7653ea9699311901dadb48816215419754f1073202: Status 404 returned error can't find the container with id f3bf1748577cf75244f62c7653ea9699311901dadb48816215419754f1073202
Oct 30 19:12:35.679551 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:35.679533    5820 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:12:36.475245 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:36.475213    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x" event={"ID":"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387","Type":"ContainerStarted","Data":"f3bf1748577cf75244f62c7653ea9699311901dadb48816215419754f1073202"}
Oct 30 19:12:37.482793 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:37.482761    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x" event={"ID":"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387","Type":"ContainerStarted","Data":"83cb33008e2c921ddbe236313c964af510c77b500ae8c25781421209bda0d5d9"}
Oct 30 19:12:37.483246 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:37.482887    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"
Oct 30 19:12:37.621202 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:37.621135    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x" podStartSLOduration=2.673638854 podStartE2EDuration="3.62111634s" totalImagesPullingTime="947.477486ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:37.62011113 +0000 UTC m=+2278.331859839" watchObservedRunningTime="2026-10-30 19:12:37.62111634 +0000 UTC m=+2278.332865040"
Oct 30 19:12:38.204592 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:12:38.204558    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:12:39.886064 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:39.884681    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:43.494511 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:43.494482    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"
Oct 30 19:12:44.827709 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:44.827670    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:45.055425 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:45.055372    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:45.228023 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:45.227925    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:49.207638 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.207603    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"]
Oct 30 19:12:49.208161 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.207893    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl" podUID="8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462" containerName="maas-api" containerID="cri-o://beb7b097b2e23f68199816bf1724dcef26e745603c9fc60c2cca98888295129c" gracePeriod=30
Oct 30 19:12:49.336370 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.336339    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:49.506288 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.506263    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"
Oct 30 19:12:49.557545 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.557518    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-openshift-service-ca\") pod \"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462\" (UID: \"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462\") "
Oct 30 19:12:49.557680 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.557597    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-kube-api-access-vkng6\" (UniqueName: \"kubernetes.io/projected/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-kube-api-access-vkng6\") pod \"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462\" (UID: \"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462\") "
Oct 30 19:12:49.557760 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.557718    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-maas-api-tls\") pod \"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462\" (UID: \"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462\") "
Oct 30 19:12:49.557804 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.557769    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-metrics-certs\") pod \"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462\" (UID: \"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462\") "
Oct 30 19:12:49.557933 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.557906    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-openshift-service-ca" pod "8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462" (UID: "8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:12:49.558617 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.558590    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:12:49.559454 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.559429    5820 generic.go:356] "Generic (PLEG): container finished" podID="8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462" containerID="beb7b097b2e23f68199816bf1724dcef26e745603c9fc60c2cca98888295129c" exitCode=0
Oct 30 19:12:49.559549 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.559477    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"
Oct 30 19:12:49.559549 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.559518    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl" event={"ID":"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462","Type":"ContainerDied","Data":"beb7b097b2e23f68199816bf1724dcef26e745603c9fc60c2cca98888295129c"}
Oct 30 19:12:49.559621 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.559574    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl" event={"ID":"8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462","Type":"ContainerDied","Data":"dc88f059b09b9531ba41dcd6b450b14b6489f224a328a7bcb1200d2cb62eb3b9"}
Oct 30 19:12:49.559621 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.559604    5820 scope.go:122] "RemoveContainer" containerID="beb7b097b2e23f68199816bf1724dcef26e745603c9fc60c2cca98888295129c"
Oct 30 19:12:49.560168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.560144    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-kube-api-access-vkng6" pod "8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462" (UID: "8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462"). InnerVolumeSpecName "kube-api-access-vkng6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:12:49.560538 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.560519    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-maas-api-tls" pod "8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462" (UID: "8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:12:49.561072 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.561046    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-metrics-certs" pod "8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462" (UID: "8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:12:49.606004 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.589043    5820 scope.go:122] "RemoveContainer" containerID="beb7b097b2e23f68199816bf1724dcef26e745603c9fc60c2cca98888295129c"
Oct 30 19:12:49.606495 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:12:49.606357    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"beb7b097b2e23f68199816bf1724dcef26e745603c9fc60c2cca98888295129c\": container with ID starting with beb7b097b2e23f68199816bf1724dcef26e745603c9fc60c2cca98888295129c not found: ID does not exist" containerID="beb7b097b2e23f68199816bf1724dcef26e745603c9fc60c2cca98888295129c"
Oct 30 19:12:49.606495 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.606409    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"beb7b097b2e23f68199816bf1724dcef26e745603c9fc60c2cca98888295129c"} err="failed to get container status \"beb7b097b2e23f68199816bf1724dcef26e745603c9fc60c2cca98888295129c\": rpc error: code = NotFound desc = could not find container \"beb7b097b2e23f68199816bf1724dcef26e745603c9fc60c2cca98888295129c\": container with ID starting with beb7b097b2e23f68199816bf1724dcef26e745603c9fc60c2cca98888295129c not found: ID does not exist"
Oct 30 19:12:49.667001 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.665192    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkng6\" (UniqueName: \"kubernetes.io/projected/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-kube-api-access-vkng6\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:12:49.667001 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.665232    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:12:49.667001 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.665247    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:12:49.989394 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.989356    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"]
Oct 30 19:12:49.997166 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:49.997135    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-83ab2c-5d586fc6c9-cgpdl"]
Oct 30 19:12:50.948374 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:50.948336    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:51.221388 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:51.221304    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:51.398546 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:51.398490    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:51.438175 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:51.438146    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462" path="/var/lib/kubelet/pods/8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462/volumes"
Oct 30 19:12:51.901895 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:51.901856    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:58.642641 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:58.642606    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:58.814598 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:58.814535    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:58.954830 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:58.954751    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"]
Oct 30 19:12:58.955087 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:58.955003    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch" podUID="9384e565-f55a-45e3-bd08-b86c27972b91" containerName="istio-proxy" containerID="cri-o://d42a61d9faf94cff5544b1f828baf70555b702c7c9e2b6d9d131044185cec452" gracePeriod=30
Oct 30 19:12:58.993440 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:58.993407    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:59.403703 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:12:59.403667    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:01.457487 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:01.457448    5820 pod_container_manager_linux.go:218] "Failed to delete cgroup paths" cgroupName=["kubepods","burstable","pod54b02e73-6065-40b6-b0cb-bc2136f9b83e"] err="unable to destroy cgroup paths for cgroup [kubepods burstable pod54b02e73-6065-40b6-b0cb-bc2136f9b83e] : Timed out while waiting for systemd to remove kubepods-burstable-pod54b02e73_6065_40b6_b0cb_bc2136f9b83e.slice"
Oct 30 19:13:01.457487 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:01.457482    5820 kubelet_pods.go:2797] "Failed to destroy orphaned pod cgroup" podUID="54b02e73-6065-40b6-b0cb-bc2136f9b83e" err="failed to delete cgroup paths for [kubepods burstable pod54b02e73-6065-40b6-b0cb-bc2136f9b83e] : unable to destroy cgroup paths for cgroup [kubepods burstable pod54b02e73-6065-40b6-b0cb-bc2136f9b83e] : Timed out while waiting for systemd to remove kubepods-burstable-pod54b02e73_6065_40b6_b0cb_bc2136f9b83e.slice"
Oct 30 19:13:04.080368 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.080055    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:04.251903 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.251873    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:13:04.363192 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.363161    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-workload-certs\") pod \"9384e565-f55a-45e3-bd08-b86c27972b91\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") "
Oct 30 19:13:04.363192 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.363197    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9384e565-f55a-45e3-bd08-b86c27972b91-istio-token\" (UniqueName: \"kubernetes.io/projected/9384e565-f55a-45e3-bd08-b86c27972b91-istio-token\") pod \"9384e565-f55a-45e3-bd08-b86c27972b91\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") "
Oct 30 19:13:04.363449 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.363263    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9384e565-f55a-45e3-bd08-b86c27972b91-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9384e565-f55a-45e3-bd08-b86c27972b91-istiod-ca-cert\") pod \"9384e565-f55a-45e3-bd08-b86c27972b91\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") "
Oct 30 19:13:04.363449 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.363305    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9384e565-f55a-45e3-bd08-b86c27972b91-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9384e565-f55a-45e3-bd08-b86c27972b91-istio-podinfo\") pod \"9384e565-f55a-45e3-bd08-b86c27972b91\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") "
Oct 30 19:13:04.363449 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.363325    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-credential-socket\") pod \"9384e565-f55a-45e3-bd08-b86c27972b91\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") "
Oct 30 19:13:04.363449 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.363355    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-istio-envoy\") pod \"9384e565-f55a-45e3-bd08-b86c27972b91\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") "
Oct 30 19:13:04.363672 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.363476    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-workload-certs" pod "9384e565-f55a-45e3-bd08-b86c27972b91" (UID: "9384e565-f55a-45e3-bd08-b86c27972b91"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:04.363672 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.363565    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-istio-data\") pod \"9384e565-f55a-45e3-bd08-b86c27972b91\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") "
Oct 30 19:13:04.363672 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.363598    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-workload-socket\") pod \"9384e565-f55a-45e3-bd08-b86c27972b91\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") "
Oct 30 19:13:04.363672 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.363642    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9384e565-f55a-45e3-bd08-b86c27972b91-kube-api-access-glcdp\" (UniqueName: \"kubernetes.io/projected/9384e565-f55a-45e3-bd08-b86c27972b91-kube-api-access-glcdp\") pod \"9384e565-f55a-45e3-bd08-b86c27972b91\" (UID: \"9384e565-f55a-45e3-bd08-b86c27972b91\") "
Oct 30 19:13:04.363893 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.363707    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9384e565-f55a-45e3-bd08-b86c27972b91-istiod-ca-cert" pod "9384e565-f55a-45e3-bd08-b86c27972b91" (UID: "9384e565-f55a-45e3-bd08-b86c27972b91"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:04.363893 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.363749    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-credential-socket" pod "9384e565-f55a-45e3-bd08-b86c27972b91" (UID: "9384e565-f55a-45e3-bd08-b86c27972b91"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:04.364101 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.363929    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-workload-socket" pod "9384e565-f55a-45e3-bd08-b86c27972b91" (UID: "9384e565-f55a-45e3-bd08-b86c27972b91"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:04.364405 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.364362    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-istio-data" pod "9384e565-f55a-45e3-bd08-b86c27972b91" (UID: "9384e565-f55a-45e3-bd08-b86c27972b91"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:04.364475 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.364434    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:04.364475 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.364458    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:04.364601 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.364474    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9384e565-f55a-45e3-bd08-b86c27972b91-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:04.364601 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.364490    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:04.365813 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.365786    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9384e565-f55a-45e3-bd08-b86c27972b91-istio-podinfo" pod "9384e565-f55a-45e3-bd08-b86c27972b91" (UID: "9384e565-f55a-45e3-bd08-b86c27972b91"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:13:04.365932 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.365829    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9384e565-f55a-45e3-bd08-b86c27972b91-istio-token" pod "9384e565-f55a-45e3-bd08-b86c27972b91" (UID: "9384e565-f55a-45e3-bd08-b86c27972b91"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:04.365932 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.365897    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-istio-envoy" pod "9384e565-f55a-45e3-bd08-b86c27972b91" (UID: "9384e565-f55a-45e3-bd08-b86c27972b91"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:04.366144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.366125    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9384e565-f55a-45e3-bd08-b86c27972b91-kube-api-access-glcdp" pod "9384e565-f55a-45e3-bd08-b86c27972b91" (UID: "9384e565-f55a-45e3-bd08-b86c27972b91"). InnerVolumeSpecName "kube-api-access-glcdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:04.465718 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.465685    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:04.465718 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.465719    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glcdp\" (UniqueName: \"kubernetes.io/projected/9384e565-f55a-45e3-bd08-b86c27972b91-kube-api-access-glcdp\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:04.465918 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.465734    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9384e565-f55a-45e3-bd08-b86c27972b91-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:04.465918 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.465747    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9384e565-f55a-45e3-bd08-b86c27972b91-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:04.465918 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.465761    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9384e565-f55a-45e3-bd08-b86c27972b91-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:04.658548 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.658515    5820 generic.go:356] "Generic (PLEG): container finished" podID="9384e565-f55a-45e3-bd08-b86c27972b91" containerID="d42a61d9faf94cff5544b1f828baf70555b702c7c9e2b6d9d131044185cec452" exitCode=0
Oct 30 19:13:04.658750 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.658567    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"
Oct 30 19:13:04.658750 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.658590    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch" event={"ID":"9384e565-f55a-45e3-bd08-b86c27972b91","Type":"ContainerDied","Data":"d42a61d9faf94cff5544b1f828baf70555b702c7c9e2b6d9d131044185cec452"}
Oct 30 19:13:04.658750 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.658640    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch" event={"ID":"9384e565-f55a-45e3-bd08-b86c27972b91","Type":"ContainerDied","Data":"00f227d2c431cd10c16faf0e4b40586791ca165f2df176c5e5d1596a5c03c06c"}
Oct 30 19:13:04.658750 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.658664    5820 scope.go:122] "RemoveContainer" containerID="d42a61d9faf94cff5544b1f828baf70555b702c7c9e2b6d9d131044185cec452"
Oct 30 19:13:04.669409 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.669389    5820 scope.go:122] "RemoveContainer" containerID="2573b5fca0645a9b31cdfd7306fb8d45beb95427a44e566994424f4e784547f9"
Oct 30 19:13:04.681366 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.681139    5820 scope.go:122] "RemoveContainer" containerID="d42a61d9faf94cff5544b1f828baf70555b702c7c9e2b6d9d131044185cec452"
Oct 30 19:13:04.683463 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:13:04.683439    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d42a61d9faf94cff5544b1f828baf70555b702c7c9e2b6d9d131044185cec452\": container with ID starting with d42a61d9faf94cff5544b1f828baf70555b702c7c9e2b6d9d131044185cec452 not found: ID does not exist" containerID="d42a61d9faf94cff5544b1f828baf70555b702c7c9e2b6d9d131044185cec452"
Oct 30 19:13:04.683576 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.683478    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d42a61d9faf94cff5544b1f828baf70555b702c7c9e2b6d9d131044185cec452"} err="failed to get container status \"d42a61d9faf94cff5544b1f828baf70555b702c7c9e2b6d9d131044185cec452\": rpc error: code = NotFound desc = could not find container \"d42a61d9faf94cff5544b1f828baf70555b702c7c9e2b6d9d131044185cec452\": container with ID starting with d42a61d9faf94cff5544b1f828baf70555b702c7c9e2b6d9d131044185cec452 not found: ID does not exist"
Oct 30 19:13:04.683576 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.683505    5820 scope.go:122] "RemoveContainer" containerID="2573b5fca0645a9b31cdfd7306fb8d45beb95427a44e566994424f4e784547f9"
Oct 30 19:13:04.683884 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:13:04.683866    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2573b5fca0645a9b31cdfd7306fb8d45beb95427a44e566994424f4e784547f9\": container with ID starting with 2573b5fca0645a9b31cdfd7306fb8d45beb95427a44e566994424f4e784547f9 not found: ID does not exist" containerID="2573b5fca0645a9b31cdfd7306fb8d45beb95427a44e566994424f4e784547f9"
Oct 30 19:13:04.683927 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.683894    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2573b5fca0645a9b31cdfd7306fb8d45beb95427a44e566994424f4e784547f9"} err="failed to get container status \"2573b5fca0645a9b31cdfd7306fb8d45beb95427a44e566994424f4e784547f9\": rpc error: code = NotFound desc = could not find container \"2573b5fca0645a9b31cdfd7306fb8d45beb95427a44e566994424f4e784547f9\": container with ID starting with 2573b5fca0645a9b31cdfd7306fb8d45beb95427a44e566994424f4e784547f9 not found: ID does not exist"
Oct 30 19:13:04.824531 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.824500    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"]
Oct 30 19:13:04.828773 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.828746    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-83ab2c-openshift-default-68856fc76f-2qvch"]
Oct 30 19:13:04.917566 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.917499    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"]
Oct 30 19:13:04.917770 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.917699    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp" podUID="cb8c3600-6aad-46a7-86a2-90f9962f461b" containerName="istio-proxy" containerID="cri-o://a448d92f5523b201dbe4acb1da275c8a506473897d4d7dee9b2311ae78debf48" gracePeriod=30
Oct 30 19:13:04.921771 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.921746    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"]
Oct 30 19:13:04.921982 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.921928    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4" podUID="a373b999-a0cb-4016-9415-e977bd058b1c" containerName="istio-proxy" containerID="cri-o://024021fc61abae1ad6a00217f2b16712f2abc1a3441d71f547761fa271efaa4a" gracePeriod=30
Oct 30 19:13:04.924329 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.924304    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"]
Oct 30 19:13:04.924505 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:04.924481    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc" podUID="fb366d57-3daf-4ffc-a3e8-e771542ae33f" containerName="istio-proxy" containerID="cri-o://4e42101efa6917d64a3e2fb8e207fe9e4761431ee955388334536a7229bdfbad" gracePeriod=30
Oct 30 19:13:05.029063 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:05.029024    5820 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-xttc4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:13:05.029276 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:05.029118    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4" podUID="a373b999-a0cb-4016-9415-e977bd058b1c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:13:05.437831 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:05.437796    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9384e565-f55a-45e3-bd08-b86c27972b91" path="/var/lib/kubelet/pods/9384e565-f55a-45e3-bd08-b86c27972b91/volumes"
Oct 30 19:13:06.074214 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:06.074183    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"]
Oct 30 19:13:06.074444 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:06.074419    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d" podUID="7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" containerName="istio-proxy" containerID="cri-o://e482831028717aa37ecb4c2c577a2b3fcd6bde8c3aaf1348bdfd5ffc6ac7d3a5" gracePeriod=30
Oct 30 19:13:08.203457 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:13:08.203410    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:13:09.626343 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:09.626287    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:10.210689 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.210621    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 19:13:10.248240 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.248211    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 19:13:10.267280 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.267254    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 19:13:10.336832 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.336807    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-workload-certs\") pod \"cb8c3600-6aad-46a7-86a2-90f9962f461b\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") "
Oct 30 19:13:10.337004 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.336868    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-workload-socket\") pod \"a373b999-a0cb-4016-9415-e977bd058b1c\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") "
Oct 30 19:13:10.337004 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.336895    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-token\" (UniqueName: \"kubernetes.io/projected/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-token\") pod \"cb8c3600-6aad-46a7-86a2-90f9962f461b\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") "
Oct 30 19:13:10.337004 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.336911    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-data\") pod \"cb8c3600-6aad-46a7-86a2-90f9962f461b\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") "
Oct 30 19:13:10.337004 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.336932    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a373b999-a0cb-4016-9415-e977bd058b1c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a373b999-a0cb-4016-9415-e977bd058b1c-istio-podinfo\") pod \"a373b999-a0cb-4016-9415-e977bd058b1c\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") "
Oct 30 19:13:10.337004 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.336976    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-workload-certs\") pod \"a373b999-a0cb-4016-9415-e977bd058b1c\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") "
Oct 30 19:13:10.337004 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.336994    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-envoy\") pod \"cb8c3600-6aad-46a7-86a2-90f9962f461b\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") "
Oct 30 19:13:10.337344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.337024    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-credential-socket\") pod \"cb8c3600-6aad-46a7-86a2-90f9962f461b\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") "
Oct 30 19:13:10.337344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.337105    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a373b999-a0cb-4016-9415-e977bd058b1c-istio-token\" (UniqueName: \"kubernetes.io/projected/a373b999-a0cb-4016-9415-e977bd058b1c-istio-token\") pod \"a373b999-a0cb-4016-9415-e977bd058b1c\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") "
Oct 30 19:13:10.337344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.337102    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-workload-certs" pod "cb8c3600-6aad-46a7-86a2-90f9962f461b" (UID: "cb8c3600-6aad-46a7-86a2-90f9962f461b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:10.337344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.337147    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cb8c3600-6aad-46a7-86a2-90f9962f461b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb8c3600-6aad-46a7-86a2-90f9962f461b-istiod-ca-cert\") pod \"cb8c3600-6aad-46a7-86a2-90f9962f461b\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") "
Oct 30 19:13:10.337344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.337179    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a373b999-a0cb-4016-9415-e977bd058b1c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a373b999-a0cb-4016-9415-e977bd058b1c-istiod-ca-cert\") pod \"a373b999-a0cb-4016-9415-e977bd058b1c\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") "
Oct 30 19:13:10.337344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.337196    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-data" pod "cb8c3600-6aad-46a7-86a2-90f9962f461b" (UID: "cb8c3600-6aad-46a7-86a2-90f9962f461b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:10.337344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.337270    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a373b999-a0cb-4016-9415-e977bd058b1c-kube-api-access-fz2qw\" (UniqueName: \"kubernetes.io/projected/a373b999-a0cb-4016-9415-e977bd058b1c-kube-api-access-fz2qw\") pod \"a373b999-a0cb-4016-9415-e977bd058b1c\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") "
Oct 30 19:13:10.337344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.337296    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-workload-socket" pod "a373b999-a0cb-4016-9415-e977bd058b1c" (UID: "a373b999-a0cb-4016-9415-e977bd058b1c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:10.337344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.337304    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-credential-socket\") pod \"a373b999-a0cb-4016-9415-e977bd058b1c\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") "
Oct 30 19:13:10.337812 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.337330    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-credential-socket" pod "cb8c3600-6aad-46a7-86a2-90f9962f461b" (UID: "cb8c3600-6aad-46a7-86a2-90f9962f461b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:10.337812 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.337365    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb8c3600-6aad-46a7-86a2-90f9962f461b-kube-api-access-hx5x9\" (UniqueName: \"kubernetes.io/projected/cb8c3600-6aad-46a7-86a2-90f9962f461b-kube-api-access-hx5x9\") pod \"cb8c3600-6aad-46a7-86a2-90f9962f461b\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") "
Oct 30 19:13:10.337812 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.337392    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-workload-socket\") pod \"cb8c3600-6aad-46a7-86a2-90f9962f461b\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") "
Oct 30 19:13:10.337812 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.337391    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-workload-certs" pod "a373b999-a0cb-4016-9415-e977bd058b1c" (UID: "a373b999-a0cb-4016-9415-e977bd058b1c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:10.337812 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.337584    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb8c3600-6aad-46a7-86a2-90f9962f461b-istiod-ca-cert" pod "cb8c3600-6aad-46a7-86a2-90f9962f461b" (UID: "cb8c3600-6aad-46a7-86a2-90f9962f461b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:10.337812 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.337700    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-istio-envoy\") pod \"a373b999-a0cb-4016-9415-e977bd058b1c\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") "
Oct 30 19:13:10.337812 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.337758    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-podinfo\") pod \"cb8c3600-6aad-46a7-86a2-90f9962f461b\" (UID: \"cb8c3600-6aad-46a7-86a2-90f9962f461b\") "
Oct 30 19:13:10.337812 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.337771    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-credential-socket" pod "a373b999-a0cb-4016-9415-e977bd058b1c" (UID: "a373b999-a0cb-4016-9415-e977bd058b1c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:10.337812 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.337809    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-istio-data\") pod \"a373b999-a0cb-4016-9415-e977bd058b1c\" (UID: \"a373b999-a0cb-4016-9415-e977bd058b1c\") "
Oct 30 19:13:10.338275 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.338077    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-workload-socket" pod "cb8c3600-6aad-46a7-86a2-90f9962f461b" (UID: "cb8c3600-6aad-46a7-86a2-90f9962f461b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:10.338385 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.338261    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a373b999-a0cb-4016-9415-e977bd058b1c-istiod-ca-cert" pod "a373b999-a0cb-4016-9415-e977bd058b1c" (UID: "a373b999-a0cb-4016-9415-e977bd058b1c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:10.339098 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.338547    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-istio-data" pod "a373b999-a0cb-4016-9415-e977bd058b1c" (UID: "a373b999-a0cb-4016-9415-e977bd058b1c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:10.339098 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.338755    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.339098 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.338786    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.339098 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.338825    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.339098 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.338872    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.339098 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.338934    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.339098 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.339003    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.339098 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.339029    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.339098 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.339043    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.339098 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.339058    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb8c3600-6aad-46a7-86a2-90f9962f461b-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.339098 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.339072    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a373b999-a0cb-4016-9415-e977bd058b1c-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.340138 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.340101    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-token" pod "cb8c3600-6aad-46a7-86a2-90f9962f461b" (UID: "cb8c3600-6aad-46a7-86a2-90f9962f461b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:10.340398 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.340370    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a373b999-a0cb-4016-9415-e977bd058b1c-istio-token" pod "a373b999-a0cb-4016-9415-e977bd058b1c" (UID: "a373b999-a0cb-4016-9415-e977bd058b1c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:10.340599 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.340578    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a373b999-a0cb-4016-9415-e977bd058b1c-istio-podinfo" pod "a373b999-a0cb-4016-9415-e977bd058b1c" (UID: "a373b999-a0cb-4016-9415-e977bd058b1c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:13:10.340683 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.340666    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-podinfo" pod "cb8c3600-6aad-46a7-86a2-90f9962f461b" (UID: "cb8c3600-6aad-46a7-86a2-90f9962f461b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:13:10.340723 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.340679    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-envoy" pod "cb8c3600-6aad-46a7-86a2-90f9962f461b" (UID: "cb8c3600-6aad-46a7-86a2-90f9962f461b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:10.340723 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.340711    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a373b999-a0cb-4016-9415-e977bd058b1c-kube-api-access-fz2qw" pod "a373b999-a0cb-4016-9415-e977bd058b1c" (UID: "a373b999-a0cb-4016-9415-e977bd058b1c"). InnerVolumeSpecName "kube-api-access-fz2qw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:10.341008 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.340936    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-istio-envoy" pod "a373b999-a0cb-4016-9415-e977bd058b1c" (UID: "a373b999-a0cb-4016-9415-e977bd058b1c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:10.341969 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.341929    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb8c3600-6aad-46a7-86a2-90f9962f461b-kube-api-access-hx5x9" pod "cb8c3600-6aad-46a7-86a2-90f9962f461b" (UID: "cb8c3600-6aad-46a7-86a2-90f9962f461b"). InnerVolumeSpecName "kube-api-access-hx5x9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:10.440417 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.440380    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-workload-certs\") pod \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") "
Oct 30 19:13:10.440604 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.440446    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-credential-socket\") pod \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") "
Oct 30 19:13:10.440604 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.440491    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istiod-ca-cert\") pod \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") "
Oct 30 19:13:10.440604 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.440529    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-token\" (UniqueName: \"kubernetes.io/projected/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-token\") pod \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") "
Oct 30 19:13:10.440604 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.440584    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb366d57-3daf-4ffc-a3e8-e771542ae33f-kube-api-access-g9pj5\" (UniqueName: \"kubernetes.io/projected/fb366d57-3daf-4ffc-a3e8-e771542ae33f-kube-api-access-g9pj5\") pod \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") "
Oct 30 19:13:10.441920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.440669    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-envoy\") pod \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") "
Oct 30 19:13:10.441920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.440723    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-workload-socket\") pod \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") "
Oct 30 19:13:10.441920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.440805    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-podinfo\") pod \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") "
Oct 30 19:13:10.441920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.440815    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-credential-socket" pod "fb366d57-3daf-4ffc-a3e8-e771542ae33f" (UID: "fb366d57-3daf-4ffc-a3e8-e771542ae33f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:10.441920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.440892    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-data\") pod \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\" (UID: \"fb366d57-3daf-4ffc-a3e8-e771542ae33f\") "
Oct 30 19:13:10.441920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.440925    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istiod-ca-cert" pod "fb366d57-3daf-4ffc-a3e8-e771542ae33f" (UID: "fb366d57-3daf-4ffc-a3e8-e771542ae33f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:10.441920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.441066    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-workload-certs" pod "fb366d57-3daf-4ffc-a3e8-e771542ae33f" (UID: "fb366d57-3daf-4ffc-a3e8-e771542ae33f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:10.441920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.441315    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-data" pod "fb366d57-3daf-4ffc-a3e8-e771542ae33f" (UID: "fb366d57-3daf-4ffc-a3e8-e771542ae33f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:10.441920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.441438    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-workload-socket" pod "fb366d57-3daf-4ffc-a3e8-e771542ae33f" (UID: "fb366d57-3daf-4ffc-a3e8-e771542ae33f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:10.441920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.441507    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.441920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.441528    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a373b999-a0cb-4016-9415-e977bd058b1c-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.441920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.441542    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.441920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.441556    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fz2qw\" (UniqueName: \"kubernetes.io/projected/a373b999-a0cb-4016-9415-e977bd058b1c-kube-api-access-fz2qw\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.441920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.441571    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx5x9\" (UniqueName: \"kubernetes.io/projected/cb8c3600-6aad-46a7-86a2-90f9962f461b-kube-api-access-hx5x9\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.441920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.441583    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a373b999-a0cb-4016-9415-e977bd058b1c-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.441920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.441599    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.441920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.441612    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.441920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.441627    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.441920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.441640    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a373b999-a0cb-4016-9415-e977bd058b1c-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.441920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.441653    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb8c3600-6aad-46a7-86a2-90f9962f461b-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.441920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.441666    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.443461 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.443120    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-podinfo" pod "fb366d57-3daf-4ffc-a3e8-e771542ae33f" (UID: "fb366d57-3daf-4ffc-a3e8-e771542ae33f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:13:10.444211 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.444183    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb366d57-3daf-4ffc-a3e8-e771542ae33f-kube-api-access-g9pj5" pod "fb366d57-3daf-4ffc-a3e8-e771542ae33f" (UID: "fb366d57-3daf-4ffc-a3e8-e771542ae33f"). InnerVolumeSpecName "kube-api-access-g9pj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:10.444341 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.444312    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-token" pod "fb366d57-3daf-4ffc-a3e8-e771542ae33f" (UID: "fb366d57-3daf-4ffc-a3e8-e771542ae33f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:10.445299 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.445278    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-envoy" pod "fb366d57-3daf-4ffc-a3e8-e771542ae33f" (UID: "fb366d57-3daf-4ffc-a3e8-e771542ae33f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:10.542918 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.542812    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.542918 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.542846    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9pj5\" (UniqueName: \"kubernetes.io/projected/fb366d57-3daf-4ffc-a3e8-e771542ae33f-kube-api-access-g9pj5\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.542918 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.542861    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.542918 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.542873    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb366d57-3daf-4ffc-a3e8-e771542ae33f-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.542918 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.542885    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb366d57-3daf-4ffc-a3e8-e771542ae33f-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.699978 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.699928    5820 generic.go:356] "Generic (PLEG): container finished" podID="a373b999-a0cb-4016-9415-e977bd058b1c" containerID="024021fc61abae1ad6a00217f2b16712f2abc1a3441d71f547761fa271efaa4a" exitCode=0
Oct 30 19:13:10.700456 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.699989    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"
Oct 30 19:13:10.700456 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.700009    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4" event={"ID":"a373b999-a0cb-4016-9415-e977bd058b1c","Type":"ContainerDied","Data":"024021fc61abae1ad6a00217f2b16712f2abc1a3441d71f547761fa271efaa4a"}
Oct 30 19:13:10.700456 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.700075    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4" event={"ID":"a373b999-a0cb-4016-9415-e977bd058b1c","Type":"ContainerDied","Data":"38c7dd829e8e9faad5f7ccb2b3881f3beef66618f4a3ed997d036173a95bc8ef"}
Oct 30 19:13:10.700456 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.700090    5820 scope.go:122] "RemoveContainer" containerID="024021fc61abae1ad6a00217f2b16712f2abc1a3441d71f547761fa271efaa4a"
Oct 30 19:13:10.702179 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.702151    5820 generic.go:356] "Generic (PLEG): container finished" podID="cb8c3600-6aad-46a7-86a2-90f9962f461b" containerID="a448d92f5523b201dbe4acb1da275c8a506473897d4d7dee9b2311ae78debf48" exitCode=0
Oct 30 19:13:10.702304 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.702210    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"
Oct 30 19:13:10.702304 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.702221    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp" event={"ID":"cb8c3600-6aad-46a7-86a2-90f9962f461b","Type":"ContainerDied","Data":"a448d92f5523b201dbe4acb1da275c8a506473897d4d7dee9b2311ae78debf48"}
Oct 30 19:13:10.702304 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.702270    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp" event={"ID":"cb8c3600-6aad-46a7-86a2-90f9962f461b","Type":"ContainerDied","Data":"d0b177f161b165ff20f15b5cb6c4f1af302f9145f1e805e1d5ece73786b437d4"}
Oct 30 19:13:10.703821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.703801    5820 generic.go:356] "Generic (PLEG): container finished" podID="fb366d57-3daf-4ffc-a3e8-e771542ae33f" containerID="4e42101efa6917d64a3e2fb8e207fe9e4761431ee955388334536a7229bdfbad" exitCode=0
Oct 30 19:13:10.703939 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.703845    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc" event={"ID":"fb366d57-3daf-4ffc-a3e8-e771542ae33f","Type":"ContainerDied","Data":"4e42101efa6917d64a3e2fb8e207fe9e4761431ee955388334536a7229bdfbad"}
Oct 30 19:13:10.703939 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.703883    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc" event={"ID":"fb366d57-3daf-4ffc-a3e8-e771542ae33f","Type":"ContainerDied","Data":"5921fc67f427b2c0b0b1e62e835d636fb5f7afcce738260fd752456b5e33dbbb"}
Oct 30 19:13:10.703939 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.703896    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"
Oct 30 19:13:10.712475 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.712459    5820 scope.go:122] "RemoveContainer" containerID="024021fc61abae1ad6a00217f2b16712f2abc1a3441d71f547761fa271efaa4a"
Oct 30 19:13:10.712767 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:13:10.712742    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"024021fc61abae1ad6a00217f2b16712f2abc1a3441d71f547761fa271efaa4a\": container with ID starting with 024021fc61abae1ad6a00217f2b16712f2abc1a3441d71f547761fa271efaa4a not found: ID does not exist" containerID="024021fc61abae1ad6a00217f2b16712f2abc1a3441d71f547761fa271efaa4a"
Oct 30 19:13:10.712861 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.712772    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"024021fc61abae1ad6a00217f2b16712f2abc1a3441d71f547761fa271efaa4a"} err="failed to get container status \"024021fc61abae1ad6a00217f2b16712f2abc1a3441d71f547761fa271efaa4a\": rpc error: code = NotFound desc = could not find container \"024021fc61abae1ad6a00217f2b16712f2abc1a3441d71f547761fa271efaa4a\": container with ID starting with 024021fc61abae1ad6a00217f2b16712f2abc1a3441d71f547761fa271efaa4a not found: ID does not exist"
Oct 30 19:13:10.712861 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.712787    5820 scope.go:122] "RemoveContainer" containerID="a448d92f5523b201dbe4acb1da275c8a506473897d4d7dee9b2311ae78debf48"
Oct 30 19:13:10.725077 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.725057    5820 scope.go:122] "RemoveContainer" containerID="a448d92f5523b201dbe4acb1da275c8a506473897d4d7dee9b2311ae78debf48"
Oct 30 19:13:10.725366 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:13:10.725346    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a448d92f5523b201dbe4acb1da275c8a506473897d4d7dee9b2311ae78debf48\": container with ID starting with a448d92f5523b201dbe4acb1da275c8a506473897d4d7dee9b2311ae78debf48 not found: ID does not exist" containerID="a448d92f5523b201dbe4acb1da275c8a506473897d4d7dee9b2311ae78debf48"
Oct 30 19:13:10.725430 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.725371    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a448d92f5523b201dbe4acb1da275c8a506473897d4d7dee9b2311ae78debf48"} err="failed to get container status \"a448d92f5523b201dbe4acb1da275c8a506473897d4d7dee9b2311ae78debf48\": rpc error: code = NotFound desc = could not find container \"a448d92f5523b201dbe4acb1da275c8a506473897d4d7dee9b2311ae78debf48\": container with ID starting with a448d92f5523b201dbe4acb1da275c8a506473897d4d7dee9b2311ae78debf48 not found: ID does not exist"
Oct 30 19:13:10.725430 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.725387    5820 scope.go:122] "RemoveContainer" containerID="4e42101efa6917d64a3e2fb8e207fe9e4761431ee955388334536a7229bdfbad"
Oct 30 19:13:10.734409 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.734392    5820 scope.go:122] "RemoveContainer" containerID="4e42101efa6917d64a3e2fb8e207fe9e4761431ee955388334536a7229bdfbad"
Oct 30 19:13:10.734696 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:13:10.734674    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e42101efa6917d64a3e2fb8e207fe9e4761431ee955388334536a7229bdfbad\": container with ID starting with 4e42101efa6917d64a3e2fb8e207fe9e4761431ee955388334536a7229bdfbad not found: ID does not exist" containerID="4e42101efa6917d64a3e2fb8e207fe9e4761431ee955388334536a7229bdfbad"
Oct 30 19:13:10.734777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.734702    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e42101efa6917d64a3e2fb8e207fe9e4761431ee955388334536a7229bdfbad"} err="failed to get container status \"4e42101efa6917d64a3e2fb8e207fe9e4761431ee955388334536a7229bdfbad\": rpc error: code = NotFound desc = could not find container \"4e42101efa6917d64a3e2fb8e207fe9e4761431ee955388334536a7229bdfbad\": container with ID starting with 4e42101efa6917d64a3e2fb8e207fe9e4761431ee955388334536a7229bdfbad not found: ID does not exist"
Oct 30 19:13:10.874409 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.874381    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"]
Oct 30 19:13:10.877326 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.877295    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-2r7zc"]
Oct 30 19:13:10.986022 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.985982    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"]
Oct 30 19:13:10.993464 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:10.993435    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-xttc4"]
Oct 30 19:13:11.111253 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.111227    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"]
Oct 30 19:13:11.117574 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.117550    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-mfkwp"]
Oct 30 19:13:11.340398 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.340368    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 19:13:11.437691 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.437664    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a373b999-a0cb-4016-9415-e977bd058b1c" path="/var/lib/kubelet/pods/a373b999-a0cb-4016-9415-e977bd058b1c/volumes"
Oct 30 19:13:11.438146 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.438131    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cb8c3600-6aad-46a7-86a2-90f9962f461b" path="/var/lib/kubelet/pods/cb8c3600-6aad-46a7-86a2-90f9962f461b/volumes"
Oct 30 19:13:11.438541 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.438529    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fb366d57-3daf-4ffc-a3e8-e771542ae33f" path="/var/lib/kubelet/pods/fb366d57-3daf-4ffc-a3e8-e771542ae33f/volumes"
Oct 30 19:13:11.455044 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.454985    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-credential-socket\") pod \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") "
Oct 30 19:13:11.455188 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.455057    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-podinfo\") pod \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") "
Oct 30 19:13:11.455188 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.455094    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-token\" (UniqueName: \"kubernetes.io/projected/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-token\") pod \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") "
Oct 30 19:13:11.455311 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.455199    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-data\") pod \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") "
Oct 30 19:13:11.455311 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.455203    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-credential-socket" pod "7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" (UID: "7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:11.455311 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.455256    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-envoy\") pod \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") "
Oct 30 19:13:11.455536 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.455315    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-kube-api-access-csd5r\" (UniqueName: \"kubernetes.io/projected/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-kube-api-access-csd5r\") pod \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") "
Oct 30 19:13:11.455536 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.455369    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-workload-socket\") pod \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") "
Oct 30 19:13:11.455536 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.455406    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-workload-certs\") pod \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") "
Oct 30 19:13:11.455536 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.455449    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istiod-ca-cert\") pod \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\" (UID: \"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c\") "
Oct 30 19:13:11.455536 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.455477    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-data" pod "7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" (UID: "7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:11.455773 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.455624    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-workload-socket" pod "7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" (UID: "7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:11.455773 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.455674    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-workload-certs" pod "7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" (UID: "7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:11.455862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.455837    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istiod-ca-cert" pod "7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" (UID: "7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:11.456011 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.455995    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:11.456089 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.456011    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:11.456089 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.456021    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:11.456089 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.456033    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:11.456089 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.456048    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:11.457469 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.457448    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-token" pod "7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" (UID: "7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:11.457822 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.457808    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-envoy" pod "7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" (UID: "7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:11.457898 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.457878    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-podinfo" pod "7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" (UID: "7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:13:11.458011 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.457989    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-kube-api-access-csd5r" pod "7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" (UID: "7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c"). InnerVolumeSpecName "kube-api-access-csd5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:11.557772 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.557742    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:11.557772 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.557767    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-csd5r\" (UniqueName: \"kubernetes.io/projected/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-kube-api-access-csd5r\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:11.557772 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.557778    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:11.558132 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.557787    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:11.712138 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.712059    5820 generic.go:356] "Generic (PLEG): container finished" podID="7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" containerID="e482831028717aa37ecb4c2c577a2b3fcd6bde8c3aaf1348bdfd5ffc6ac7d3a5" exitCode=0
Oct 30 19:13:11.712138 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.712106    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"
Oct 30 19:13:11.712635 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.712145    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d" event={"ID":"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c","Type":"ContainerDied","Data":"e482831028717aa37ecb4c2c577a2b3fcd6bde8c3aaf1348bdfd5ffc6ac7d3a5"}
Oct 30 19:13:11.712635 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.712201    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d" event={"ID":"7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c","Type":"ContainerDied","Data":"360cc63849a2b6c2ab9e0a69b0a6dcf21874e18366235dbf15965610f3223e32"}
Oct 30 19:13:11.712635 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.712235    5820 scope.go:122] "RemoveContainer" containerID="e482831028717aa37ecb4c2c577a2b3fcd6bde8c3aaf1348bdfd5ffc6ac7d3a5"
Oct 30 19:13:11.722684 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.722663    5820 scope.go:122] "RemoveContainer" containerID="5719f491e289737cf0b5b7750f69b79e4bf869a90122f9d13f5a6ecbe0589ad6"
Oct 30 19:13:11.731400 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.731383    5820 scope.go:122] "RemoveContainer" containerID="e482831028717aa37ecb4c2c577a2b3fcd6bde8c3aaf1348bdfd5ffc6ac7d3a5"
Oct 30 19:13:11.731642 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:13:11.731623    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e482831028717aa37ecb4c2c577a2b3fcd6bde8c3aaf1348bdfd5ffc6ac7d3a5\": container with ID starting with e482831028717aa37ecb4c2c577a2b3fcd6bde8c3aaf1348bdfd5ffc6ac7d3a5 not found: ID does not exist" containerID="e482831028717aa37ecb4c2c577a2b3fcd6bde8c3aaf1348bdfd5ffc6ac7d3a5"
Oct 30 19:13:11.731706 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.731650    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e482831028717aa37ecb4c2c577a2b3fcd6bde8c3aaf1348bdfd5ffc6ac7d3a5"} err="failed to get container status \"e482831028717aa37ecb4c2c577a2b3fcd6bde8c3aaf1348bdfd5ffc6ac7d3a5\": rpc error: code = NotFound desc = could not find container \"e482831028717aa37ecb4c2c577a2b3fcd6bde8c3aaf1348bdfd5ffc6ac7d3a5\": container with ID starting with e482831028717aa37ecb4c2c577a2b3fcd6bde8c3aaf1348bdfd5ffc6ac7d3a5 not found: ID does not exist"
Oct 30 19:13:11.731706 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.731670    5820 scope.go:122] "RemoveContainer" containerID="5719f491e289737cf0b5b7750f69b79e4bf869a90122f9d13f5a6ecbe0589ad6"
Oct 30 19:13:11.731906 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:13:11.731890    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5719f491e289737cf0b5b7750f69b79e4bf869a90122f9d13f5a6ecbe0589ad6\": container with ID starting with 5719f491e289737cf0b5b7750f69b79e4bf869a90122f9d13f5a6ecbe0589ad6 not found: ID does not exist" containerID="5719f491e289737cf0b5b7750f69b79e4bf869a90122f9d13f5a6ecbe0589ad6"
Oct 30 19:13:11.731965 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.731913    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5719f491e289737cf0b5b7750f69b79e4bf869a90122f9d13f5a6ecbe0589ad6"} err="failed to get container status \"5719f491e289737cf0b5b7750f69b79e4bf869a90122f9d13f5a6ecbe0589ad6\": rpc error: code = NotFound desc = could not find container \"5719f491e289737cf0b5b7750f69b79e4bf869a90122f9d13f5a6ecbe0589ad6\": container with ID starting with 5719f491e289737cf0b5b7750f69b79e4bf869a90122f9d13f5a6ecbe0589ad6 not found: ID does not exist"
Oct 30 19:13:11.880449 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.880417    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"]
Oct 30 19:13:11.883753 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:11.883719    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-5dd6d"]
Oct 30 19:13:13.437388 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:13.437349    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" path="/var/lib/kubelet/pods/7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c/volumes"
Oct 30 19:13:14.799398 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.799363    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"]
Oct 30 19:13:14.799927 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.799913    5820 cpu_manager.go:414] "containerMap: removing container" podUID="9384e565-f55a-45e3-bd08-b86c27972b91" containerName="istio-proxy"
Oct 30 19:13:14.800034 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.799928    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9384e565-f55a-45e3-bd08-b86c27972b91" containerName="istio-proxy"
Oct 30 19:13:14.800034 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.799982    5820 cpu_manager.go:414] "containerMap: removing container" podUID="8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462" containerName="maas-api"
Oct 30 19:13:14.800034 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.799990    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462" containerName="maas-api"
Oct 30 19:13:14.800034 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.800004    5820 cpu_manager.go:414] "containerMap: removing container" podUID="7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" containerName="istio-proxy"
Oct 30 19:13:14.800034 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.800012    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" containerName="istio-proxy"
Oct 30 19:13:14.800034 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.800029    5820 cpu_manager.go:414] "containerMap: removing container" podUID="fb366d57-3daf-4ffc-a3e8-e771542ae33f" containerName="istio-proxy"
Oct 30 19:13:14.800034 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.800034    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb366d57-3daf-4ffc-a3e8-e771542ae33f" containerName="istio-proxy"
Oct 30 19:13:14.800258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.800044    5820 cpu_manager.go:414] "containerMap: removing container" podUID="cb8c3600-6aad-46a7-86a2-90f9962f461b" containerName="istio-proxy"
Oct 30 19:13:14.800258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.800049    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb8c3600-6aad-46a7-86a2-90f9962f461b" containerName="istio-proxy"
Oct 30 19:13:14.800258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.800060    5820 cpu_manager.go:414] "containerMap: removing container" podUID="7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" containerName="istio-proxy"
Oct 30 19:13:14.800258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.800066    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" containerName="istio-proxy"
Oct 30 19:13:14.800258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.800078    5820 cpu_manager.go:414] "containerMap: removing container" podUID="a373b999-a0cb-4016-9415-e977bd058b1c" containerName="istio-proxy"
Oct 30 19:13:14.800258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.800082    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a373b999-a0cb-4016-9415-e977bd058b1c" containerName="istio-proxy"
Oct 30 19:13:14.800258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.800180    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="8ca42ea4-0b3c-424e-ba8e-7f6cd7e00462" containerName="maas-api"
Oct 30 19:13:14.800258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.800191    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" containerName="istio-proxy"
Oct 30 19:13:14.800258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.800198    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb366d57-3daf-4ffc-a3e8-e771542ae33f" containerName="istio-proxy"
Oct 30 19:13:14.800258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.800205    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb8c3600-6aad-46a7-86a2-90f9962f461b" containerName="istio-proxy"
Oct 30 19:13:14.800258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.800210    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="9384e565-f55a-45e3-bd08-b86c27972b91" containerName="istio-proxy"
Oct 30 19:13:14.800258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.800215    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d8121e9-2b0a-4114-ba0e-5b9b8644fb0c" containerName="istio-proxy"
Oct 30 19:13:14.800258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.800221    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="a373b999-a0cb-4016-9415-e977bd058b1c" containerName="istio-proxy"
Oct 30 19:13:14.800258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.800227    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="9384e565-f55a-45e3-bd08-b86c27972b91" containerName="istio-proxy"
Oct 30 19:13:14.805306 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.805286    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"
Oct 30 19:13:14.930019 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.928642    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"]
Oct 30 19:13:14.930019 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.928753    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-cf43e7-serving-cert\""
Oct 30 19:13:14.994626 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.994589    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3b24a33-1a96-41be-abca-2a75550b1c46-metrics-certs\") pod \"maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw\" (UID: \"c3b24a33-1a96-41be-abca-2a75550b1c46\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"
Oct 30 19:13:14.994774 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.994696    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3b24a33-1a96-41be-abca-2a75550b1c46-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw\" (UID: \"c3b24a33-1a96-41be-abca-2a75550b1c46\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"
Oct 30 19:13:14.994774 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.994747    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbcrn\" (UniqueName: \"kubernetes.io/projected/c3b24a33-1a96-41be-abca-2a75550b1c46-kube-api-access-gbcrn\") pod \"maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw\" (UID: \"c3b24a33-1a96-41be-abca-2a75550b1c46\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"
Oct 30 19:13:14.994889 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:14.994802    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3b24a33-1a96-41be-abca-2a75550b1c46-maas-api-tls\") pod \"maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw\" (UID: \"c3b24a33-1a96-41be-abca-2a75550b1c46\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"
Oct 30 19:13:15.083842 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:15.083763    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:15.095902 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:15.095877    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3b24a33-1a96-41be-abca-2a75550b1c46-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw\" (UID: \"c3b24a33-1a96-41be-abca-2a75550b1c46\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"
Oct 30 19:13:15.096063 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:15.095919    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbcrn\" (UniqueName: \"kubernetes.io/projected/c3b24a33-1a96-41be-abca-2a75550b1c46-kube-api-access-gbcrn\") pod \"maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw\" (UID: \"c3b24a33-1a96-41be-abca-2a75550b1c46\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"
Oct 30 19:13:15.096063 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:15.096034    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3b24a33-1a96-41be-abca-2a75550b1c46-maas-api-tls\") pod \"maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw\" (UID: \"c3b24a33-1a96-41be-abca-2a75550b1c46\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"
Oct 30 19:13:15.096165 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:15.096151    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3b24a33-1a96-41be-abca-2a75550b1c46-metrics-certs\") pod \"maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw\" (UID: \"c3b24a33-1a96-41be-abca-2a75550b1c46\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"
Oct 30 19:13:15.096684 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:15.096666    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3b24a33-1a96-41be-abca-2a75550b1c46-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw\" (UID: \"c3b24a33-1a96-41be-abca-2a75550b1c46\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"
Oct 30 19:13:15.098588 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:15.098566    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3b24a33-1a96-41be-abca-2a75550b1c46-maas-api-tls\") pod \"maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw\" (UID: \"c3b24a33-1a96-41be-abca-2a75550b1c46\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"
Oct 30 19:13:15.098706 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:15.098627    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3b24a33-1a96-41be-abca-2a75550b1c46-metrics-certs\") pod \"maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw\" (UID: \"c3b24a33-1a96-41be-abca-2a75550b1c46\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"
Oct 30 19:13:15.159253 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:15.159218    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbcrn\" (UniqueName: \"kubernetes.io/projected/c3b24a33-1a96-41be-abca-2a75550b1c46-kube-api-access-gbcrn\") pod \"maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw\" (UID: \"c3b24a33-1a96-41be-abca-2a75550b1c46\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"
Oct 30 19:13:15.265013 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:15.264943    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:15.418997 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:15.418890    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"
Oct 30 19:13:15.491975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:15.489849    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:15.823058 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:15.823029    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"]
Oct 30 19:13:15.824706 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:13:15.824674    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3b24a33_1a96_41be_abca_2a75550b1c46.slice/crio-830427731e81fcd8f56dc796ce422d216c660c4c0a241f3c12621363ed262b82 WatchSource:0}: Error finding container 830427731e81fcd8f56dc796ce422d216c660c4c0a241f3c12621363ed262b82: Status 404 returned error can't find the container with id 830427731e81fcd8f56dc796ce422d216c660c4c0a241f3c12621363ed262b82
Oct 30 19:13:16.662265 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:16.662231    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"]
Oct 30 19:13:16.662514 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:16.662484    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz" podUID="6a253f0b-1803-4c70-b5af-aa295e8c47cb" containerName="istio-proxy" containerID="cri-o://b777c3733a3fd93aa0d2b8d0118c99b2216857b3f965ee87d745e3f607ebd8ec" gracePeriod=30
Oct 30 19:13:16.775303 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:16.775267    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw" event={"ID":"c3b24a33-1a96-41be-abca-2a75550b1c46","Type":"ContainerStarted","Data":"830427731e81fcd8f56dc796ce422d216c660c4c0a241f3c12621363ed262b82"}
Oct 30 19:13:17.026583 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:17.026485    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9"]
Oct 30 19:13:17.413529 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:17.413494    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:17.624050 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:17.624014    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:17.783514 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:17.783419    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw" event={"ID":"c3b24a33-1a96-41be-abca-2a75550b1c46","Type":"ContainerStarted","Data":"016faa8c3b57b74651c932e32835d1aa47bd4ad281698bec4f0fbb230e5f378c"}
Oct 30 19:13:17.783514 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:17.783479    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"
Oct 30 19:13:17.926272 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:17.926206    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw" podStartSLOduration=3.06013895 podStartE2EDuration="3.921721913s" totalImagesPullingTime="861.582963ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:13:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:13:17.920904391 +0000 UTC m=+2318.632653197" watchObservedRunningTime="2026-10-30 19:13:17.921721913 +0000 UTC m=+2318.633470612"
Oct 30 19:13:20.093500 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:20.088614    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:21.811882 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:21.811842    5820 generic.go:356] "Generic (PLEG): container finished" podID="6a253f0b-1803-4c70-b5af-aa295e8c47cb" containerID="b777c3733a3fd93aa0d2b8d0118c99b2216857b3f965ee87d745e3f607ebd8ec" exitCode=0
Oct 30 19:13:21.812304 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:21.811888    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz" event={"ID":"6a253f0b-1803-4c70-b5af-aa295e8c47cb","Type":"ContainerDied","Data":"b777c3733a3fd93aa0d2b8d0118c99b2216857b3f965ee87d745e3f607ebd8ec"}
Oct 30 19:13:21.812304 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:21.811980    5820 scope.go:122] "RemoveContainer" containerID="1e8c99c5b86a3fde4d5718e0bf838f84c9e76a8dcd5ae60f6cd13e524a2a4e87"
Oct 30 19:13:21.947182 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:21.947124    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 19:13:22.027392 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.027348    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9" podUID="011ebb2b-f670-4df6-aff6-1c419f38cb1d" containerName="payload-pre-processing" containerID="cri-o://73ad5c5e76c94528797bb58054c4a35dffe6a6f0d8d8c94b125a97877693ce3b" gracePeriod=25
Oct 30 19:13:22.032378 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.032350    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a253f0b-1803-4c70-b5af-aa295e8c47cb-kube-api-access-t7tz8\" (UniqueName: \"kubernetes.io/projected/6a253f0b-1803-4c70-b5af-aa295e8c47cb-kube-api-access-t7tz8\") pod \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") "
Oct 30 19:13:22.032560 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.032520    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-workload-socket\") pod \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") "
Oct 30 19:13:22.032635 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.032561    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-envoy\") pod \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") "
Oct 30 19:13:22.032635 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.032611    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-podinfo\") pod \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") "
Oct 30 19:13:22.032716 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.032650    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-workload-certs\") pod \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") "
Oct 30 19:13:22.032770 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.032714    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istiod-ca-cert\") pod \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") "
Oct 30 19:13:22.032808 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.032778    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-data\") pod \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") "
Oct 30 19:13:22.032853 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.032810    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-token\" (UniqueName: \"kubernetes.io/projected/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-token\") pod \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") "
Oct 30 19:13:22.032894 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.032856    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-credential-socket\") pod \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\" (UID: \"6a253f0b-1803-4c70-b5af-aa295e8c47cb\") "
Oct 30 19:13:22.034153 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.033812    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istiod-ca-cert" pod "6a253f0b-1803-4c70-b5af-aa295e8c47cb" (UID: "6a253f0b-1803-4c70-b5af-aa295e8c47cb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:22.034153 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.033934    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-workload-socket" pod "6a253f0b-1803-4c70-b5af-aa295e8c47cb" (UID: "6a253f0b-1803-4c70-b5af-aa295e8c47cb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:22.034153 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.034103    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-data" pod "6a253f0b-1803-4c70-b5af-aa295e8c47cb" (UID: "6a253f0b-1803-4c70-b5af-aa295e8c47cb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:22.034153 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.034118    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-workload-certs" pod "6a253f0b-1803-4c70-b5af-aa295e8c47cb" (UID: "6a253f0b-1803-4c70-b5af-aa295e8c47cb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:22.034476 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.034159    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-credential-socket" pod "6a253f0b-1803-4c70-b5af-aa295e8c47cb" (UID: "6a253f0b-1803-4c70-b5af-aa295e8c47cb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:22.035225 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.035193    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-podinfo" pod "6a253f0b-1803-4c70-b5af-aa295e8c47cb" (UID: "6a253f0b-1803-4c70-b5af-aa295e8c47cb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:13:22.035587 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.035569    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-envoy" pod "6a253f0b-1803-4c70-b5af-aa295e8c47cb" (UID: "6a253f0b-1803-4c70-b5af-aa295e8c47cb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:22.036030 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.036011    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a253f0b-1803-4c70-b5af-aa295e8c47cb-kube-api-access-t7tz8" pod "6a253f0b-1803-4c70-b5af-aa295e8c47cb" (UID: "6a253f0b-1803-4c70-b5af-aa295e8c47cb"). InnerVolumeSpecName "kube-api-access-t7tz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:22.036236 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.036207    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-token" pod "6a253f0b-1803-4c70-b5af-aa295e8c47cb" (UID: "6a253f0b-1803-4c70-b5af-aa295e8c47cb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:22.136039 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.136003    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7tz8\" (UniqueName: \"kubernetes.io/projected/6a253f0b-1803-4c70-b5af-aa295e8c47cb-kube-api-access-t7tz8\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:22.136039 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.136040    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:22.136039 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.136056    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:22.136346 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.136068    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:22.136346 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.136077    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:22.136346 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.136085    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:22.136346 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.136093    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:22.136346 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.136106    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a253f0b-1803-4c70-b5af-aa295e8c47cb-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:22.136346 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.136118    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a253f0b-1803-4c70-b5af-aa295e8c47cb-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:22.298095 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.298067    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9"
Oct 30 19:13:22.339208 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.339174    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/011ebb2b-f670-4df6-aff6-1c419f38cb1d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/011ebb2b-f670-4df6-aff6-1c419f38cb1d-plugins-config-volume\") pod \"011ebb2b-f670-4df6-aff6-1c419f38cb1d\" (UID: \"011ebb2b-f670-4df6-aff6-1c419f38cb1d\") "
Oct 30 19:13:22.339391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.339333    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/011ebb2b-f670-4df6-aff6-1c419f38cb1d-kube-api-access-c26wk\" (UniqueName: \"kubernetes.io/projected/011ebb2b-f670-4df6-aff6-1c419f38cb1d-kube-api-access-c26wk\") pod \"011ebb2b-f670-4df6-aff6-1c419f38cb1d\" (UID: \"011ebb2b-f670-4df6-aff6-1c419f38cb1d\") "
Oct 30 19:13:22.339593 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.339568    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/011ebb2b-f670-4df6-aff6-1c419f38cb1d-plugins-config-volume" pod "011ebb2b-f670-4df6-aff6-1c419f38cb1d" (UID: "011ebb2b-f670-4df6-aff6-1c419f38cb1d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:22.339931 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.339898    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/011ebb2b-f670-4df6-aff6-1c419f38cb1d-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:22.341678 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.341655    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/011ebb2b-f670-4df6-aff6-1c419f38cb1d-kube-api-access-c26wk" pod "011ebb2b-f670-4df6-aff6-1c419f38cb1d" (UID: "011ebb2b-f670-4df6-aff6-1c419f38cb1d"). InnerVolumeSpecName "kube-api-access-c26wk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:22.441167 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.441127    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c26wk\" (UniqueName: \"kubernetes.io/projected/011ebb2b-f670-4df6-aff6-1c419f38cb1d-kube-api-access-c26wk\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:22.820763 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.820729    5820 generic.go:356] "Generic (PLEG): container finished" podID="011ebb2b-f670-4df6-aff6-1c419f38cb1d" containerID="73ad5c5e76c94528797bb58054c4a35dffe6a6f0d8d8c94b125a97877693ce3b" exitCode=0
Oct 30 19:13:22.821308 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.820787    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9"
Oct 30 19:13:22.821308 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.820830    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9" event={"ID":"011ebb2b-f670-4df6-aff6-1c419f38cb1d","Type":"ContainerDied","Data":"73ad5c5e76c94528797bb58054c4a35dffe6a6f0d8d8c94b125a97877693ce3b"}
Oct 30 19:13:22.821308 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.820888    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9" event={"ID":"011ebb2b-f670-4df6-aff6-1c419f38cb1d","Type":"ContainerDied","Data":"b07398b702da993c5df7f6ef9efd95c6e99c0fee13a317eafaae6feb202599d6"}
Oct 30 19:13:22.821308 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.820916    5820 scope.go:122] "RemoveContainer" containerID="73ad5c5e76c94528797bb58054c4a35dffe6a6f0d8d8c94b125a97877693ce3b"
Oct 30 19:13:22.822785 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.822755    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz" event={"ID":"6a253f0b-1803-4c70-b5af-aa295e8c47cb","Type":"ContainerDied","Data":"ff87535ea84dc8ea4c50e4da19783cecd6d4b66af7fae6eff7b490d0b462cc4c"}
Oct 30 19:13:22.822912 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.822788    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"
Oct 30 19:13:22.837138 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.836360    5820 scope.go:122] "RemoveContainer" containerID="73ad5c5e76c94528797bb58054c4a35dffe6a6f0d8d8c94b125a97877693ce3b"
Oct 30 19:13:22.837138 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:13:22.837057    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73ad5c5e76c94528797bb58054c4a35dffe6a6f0d8d8c94b125a97877693ce3b\": container with ID starting with 73ad5c5e76c94528797bb58054c4a35dffe6a6f0d8d8c94b125a97877693ce3b not found: ID does not exist" containerID="73ad5c5e76c94528797bb58054c4a35dffe6a6f0d8d8c94b125a97877693ce3b"
Oct 30 19:13:22.837138 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.837094    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73ad5c5e76c94528797bb58054c4a35dffe6a6f0d8d8c94b125a97877693ce3b"} err="failed to get container status \"73ad5c5e76c94528797bb58054c4a35dffe6a6f0d8d8c94b125a97877693ce3b\": rpc error: code = NotFound desc = could not find container \"73ad5c5e76c94528797bb58054c4a35dffe6a6f0d8d8c94b125a97877693ce3b\": container with ID starting with 73ad5c5e76c94528797bb58054c4a35dffe6a6f0d8d8c94b125a97877693ce3b not found: ID does not exist"
Oct 30 19:13:22.837138 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.837117    5820 scope.go:122] "RemoveContainer" containerID="b777c3733a3fd93aa0d2b8d0118c99b2216857b3f965ee87d745e3f607ebd8ec"
Oct 30 19:13:22.893542 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:13:22.893508    5820 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a253f0b_1803_4c70_b5af_aa295e8c47cb.slice/crio-ff87535ea84dc8ea4c50e4da19783cecd6d4b66af7fae6eff7b490d0b462cc4c\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a253f0b_1803_4c70_b5af_aa295e8c47cb.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod011ebb2b_f670_4df6_aff6_1c419f38cb1d.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod011ebb2b_f670_4df6_aff6_1c419f38cb1d.slice/crio-b07398b702da993c5df7f6ef9efd95c6e99c0fee13a317eafaae6feb202599d6\": RecentStats: unable to find data in memory cache]"
Oct 30 19:13:22.985274 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.985196    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9"]
Oct 30 19:13:22.990118 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:22.990090    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-5a035d-5b5cff4cf9-8s6t9"]
Oct 30 19:13:23.100109 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:23.100072    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"]
Oct 30 19:13:23.104514 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:23.104485    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-vxlhz"]
Oct 30 19:13:23.437653 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:23.437621    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="011ebb2b-f670-4df6-aff6-1c419f38cb1d" path="/var/lib/kubelet/pods/011ebb2b-f670-4df6-aff6-1c419f38cb1d/volumes"
Oct 30 19:13:23.438102 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:23.438085    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a253f0b-1803-4c70-b5af-aa295e8c47cb" path="/var/lib/kubelet/pods/6a253f0b-1803-4c70-b5af-aa295e8c47cb/volumes"
Oct 30 19:13:23.796903 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:23.796806    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"
Oct 30 19:13:24.844021 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:24.843164    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:24.852338 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:24.852309    5820 generic.go:356] "Generic (PLEG): container finished" podID="31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" containerID="443e08ecf06fa9576b471d7ac215f5aff6534ae8cc9faf58c00847e117a32811" exitCode=0
Oct 30 19:13:24.852518 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:24.852386    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx" event={"ID":"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9","Type":"ContainerDied","Data":"443e08ecf06fa9576b471d7ac215f5aff6534ae8cc9faf58c00847e117a32811"}
Oct 30 19:13:24.852823 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:24.852806    5820 scope.go:122] "RemoveContainer" containerID="443e08ecf06fa9576b471d7ac215f5aff6534ae8cc9faf58c00847e117a32811"
Oct 30 19:13:24.854291 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:24.854263    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:13:24.854366 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:24.854332    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:13:24.854415 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:24.854366    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:13:25.084139 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:25.084110    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:13:25.860743 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:25.860708    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx" event={"ID":"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9","Type":"ContainerStarted","Data":"ff3490f62ae959e8616aee05b2cd4cfb1d1feaeb669ec868f8d386e6cd096c6d"}
Oct 30 19:13:26.860694 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:26.860658    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:13:26.862240 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:26.862207    5820 patch_prober.go:29] interesting pod/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 19:13:26.862744 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:26.862268    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx" podUID="31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 19:13:27.860555 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:27.860515    5820 patch_prober.go:29] interesting pod/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 19:13:27.860772 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:27.860586    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx" podUID="31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 19:13:28.860667 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:28.860620    5820 patch_prober.go:29] interesting pod/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 19:13:28.861122 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:28.860684    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx" podUID="31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 19:13:29.671621 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:29.671572    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:29.860336 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:29.860300    5820 patch_prober.go:29] interesting pod/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 19:13:29.860533 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:29.860369    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx" podUID="31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 19:13:30.860912 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:30.860866    5820 patch_prober.go:29] interesting pod/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 19:13:30.861435 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:30.860937    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx" podUID="31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 19:13:31.864613 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:31.864585    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:13:31.865067 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:31.864857    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:13:31.865636 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:31.865617    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:13:34.556987 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:34.556932    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:38.205911 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:13:38.205872    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:13:39.317185 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:39.317157    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-7kxzm_88b8cb4c-656b-44e4-8632-0acff65c58ad/payload-processing/0.log"
Oct 30 19:13:39.586748 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:39.586719    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-7kxzm_88b8cb4c-656b-44e4-8632-0acff65c58ad/payload-processing/0.log"
Oct 30 19:13:39.660817 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:39.660764    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:42.983743 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:42.983706    5820 generic.go:356] "Generic (PLEG): container finished" podID="fd708157-c511-4334-84c0-6d0a655e3502" containerID="6c51e3a8f0375ca46d81242e388b2c87378f348315273f154900a53e407d64cc" exitCode=0
Oct 30 19:13:42.984296 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:42.983780    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq" event={"ID":"fd708157-c511-4334-84c0-6d0a655e3502","Type":"ContainerDied","Data":"6c51e3a8f0375ca46d81242e388b2c87378f348315273f154900a53e407d64cc"}
Oct 30 19:13:42.984296 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:42.984223    5820 scope.go:122] "RemoveContainer" containerID="6c51e3a8f0375ca46d81242e388b2c87378f348315273f154900a53e407d64cc"
Oct 30 19:13:42.985746 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:42.985717    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:13:42.985812 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:42.985790    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:13:42.985852 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:42.985826    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:13:43.991840 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:43.991806    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq" event={"ID":"fd708157-c511-4334-84c0-6d0a655e3502","Type":"ContainerStarted","Data":"66e767bb4219d89e63c9b136536add3a1cd8521432e97983928a8f60aace9844"}
Oct 30 19:13:44.332600 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:44.332568    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:13:44.332874 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:44.332852    5820 patch_prober.go:29] interesting pod/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:13:44.332962 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:44.332901    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq" podUID="fd708157-c511-4334-84c0-6d0a655e3502" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 19:13:45.332851 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:45.332814    5820 patch_prober.go:29] interesting pod/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:13:45.333258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:45.332868    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq" podUID="fd708157-c511-4334-84c0-6d0a655e3502" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 19:13:46.332677 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:46.332636    5820 patch_prober.go:29] interesting pod/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:13:46.332884 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:46.332696    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq" podUID="fd708157-c511-4334-84c0-6d0a655e3502" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 19:13:46.557655 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:46.557616    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:47.333030 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:47.332932    5820 patch_prober.go:29] interesting pod/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:13:47.333030 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:47.333005    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq" podUID="fd708157-c511-4334-84c0-6d0a655e3502" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 19:13:48.332397 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:48.332359    5820 patch_prober.go:29] interesting pod/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:13:48.332590 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:48.332425    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq" podUID="fd708157-c511-4334-84c0-6d0a655e3502" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 19:13:49.332234 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:49.332196    5820 patch_prober.go:29] interesting pod/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:13:49.332642 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:49.332248    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq" podUID="fd708157-c511-4334-84c0-6d0a655e3502" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 19:13:50.336678 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:50.336648    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:13:50.337091 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:50.337066    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:13:50.337667 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:50.337648    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:13:52.028467 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:52.028430    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:54.818869 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:54.818828    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:56.065570 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.065529    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:56.604683 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.604651    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"]
Oct 30 19:13:56.605280 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.605266    5820 cpu_manager.go:414] "containerMap: removing container" podUID="011ebb2b-f670-4df6-aff6-1c419f38cb1d" containerName="payload-pre-processing"
Oct 30 19:13:56.605340 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.605281    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="011ebb2b-f670-4df6-aff6-1c419f38cb1d" containerName="payload-pre-processing"
Oct 30 19:13:56.605340 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.605313    5820 cpu_manager.go:414] "containerMap: removing container" podUID="6a253f0b-1803-4c70-b5af-aa295e8c47cb" containerName="istio-proxy"
Oct 30 19:13:56.605340 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.605319    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a253f0b-1803-4c70-b5af-aa295e8c47cb" containerName="istio-proxy"
Oct 30 19:13:56.605340 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.605326    5820 cpu_manager.go:414] "containerMap: removing container" podUID="9384e565-f55a-45e3-bd08-b86c27972b91" containerName="istio-proxy"
Oct 30 19:13:56.605340 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.605331    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9384e565-f55a-45e3-bd08-b86c27972b91" containerName="istio-proxy"
Oct 30 19:13:56.605528 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.605345    5820 cpu_manager.go:414] "containerMap: removing container" podUID="6a253f0b-1803-4c70-b5af-aa295e8c47cb" containerName="istio-proxy"
Oct 30 19:13:56.605528 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.605349    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a253f0b-1803-4c70-b5af-aa295e8c47cb" containerName="istio-proxy"
Oct 30 19:13:56.605528 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.605445    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="011ebb2b-f670-4df6-aff6-1c419f38cb1d" containerName="payload-pre-processing"
Oct 30 19:13:56.605528 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.605458    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a253f0b-1803-4c70-b5af-aa295e8c47cb" containerName="istio-proxy"
Oct 30 19:13:56.605528 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.605472    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a253f0b-1803-4c70-b5af-aa295e8c47cb" containerName="istio-proxy"
Oct 30 19:13:56.609344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.609328    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.666360 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.666332    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-6d96a6-openshift-default-dockercfg-ddm8c\""
Oct 30 19:13:56.696582 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.693724    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/126f2139-f087-4162-af9d-ca9edd5c83da-istio-token\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.697036 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.697007    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-istio-data\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.697169 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.697055    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj4sv\" (UniqueName: \"kubernetes.io/projected/126f2139-f087-4162-af9d-ca9edd5c83da-kube-api-access-bj4sv\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.697169 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.697109    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/126f2139-f087-4162-af9d-ca9edd5c83da-istio-podinfo\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.697169 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.697147    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-workload-socket\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.697284 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.697170    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-workload-certs\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.697284 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.697234    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-istio-envoy\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.697284 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.697266    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/126f2139-f087-4162-af9d-ca9edd5c83da-istiod-ca-cert\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.697413 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.697361    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-credential-socket\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.718249 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.718214    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"]
Oct 30 19:13:56.738575 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.738543    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:56.801471 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.801433    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/126f2139-f087-4162-af9d-ca9edd5c83da-istio-podinfo\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.801719 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.801696    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-workload-socket\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.805002 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.802151    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-workload-socket\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.805002 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.802510    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-workload-certs\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.805002 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.802569    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-workload-certs\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.805002 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.802666    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-istio-envoy\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.805002 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.803615    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/126f2139-f087-4162-af9d-ca9edd5c83da-istiod-ca-cert\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.805338 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.802701    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/126f2139-f087-4162-af9d-ca9edd5c83da-istiod-ca-cert\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.805338 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.805144    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-credential-socket\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.805488 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.805452    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-credential-socket\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.812585 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.812546    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-istio-envoy\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.812711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.812608    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/126f2139-f087-4162-af9d-ca9edd5c83da-istio-token\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.812711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.812653    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-istio-data\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.812711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.812688    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj4sv\" (UniqueName: \"kubernetes.io/projected/126f2139-f087-4162-af9d-ca9edd5c83da-kube-api-access-bj4sv\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.813034 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.813011    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-istio-data\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.813131 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.813052    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/126f2139-f087-4162-af9d-ca9edd5c83da-istio-podinfo\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.876182 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.876103    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj4sv\" (UniqueName: \"kubernetes.io/projected/126f2139-f087-4162-af9d-ca9edd5c83da-kube-api-access-bj4sv\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.879870 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.879834    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/126f2139-f087-4162-af9d-ca9edd5c83da-istio-token\") pod \"e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") " pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.945079 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.945041    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:56.975801 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:56.975766    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:57.270030 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:57.264419    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:57.376160 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:57.376136    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"]
Oct 30 19:13:57.378077 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:13:57.378044    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod126f2139_f087_4162_af9d_ca9edd5c83da.slice/crio-6769174f8c703a056130a8d0740253e8b56ad60fee1a5767283d8d9d3b4bd1d1 WatchSource:0}: Error finding container 6769174f8c703a056130a8d0740253e8b56ad60fee1a5767283d8d9d3b4bd1d1: Status 404 returned error can't find the container with id 6769174f8c703a056130a8d0740253e8b56ad60fee1a5767283d8d9d3b4bd1d1
Oct 30 19:13:57.380543 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:57.380498    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:13:57.380634 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:57.380570    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:13:57.380634 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:57.380596    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:13:58.093920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:58.093883    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b" event={"ID":"126f2139-f087-4162-af9d-ca9edd5c83da","Type":"ContainerStarted","Data":"2531213a74bf757ed94e1886fbd2cf3e7bc04993c5c4a70823a4401f767f3e6c"}
Oct 30 19:13:58.094155 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:58.093975    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b" event={"ID":"126f2139-f087-4162-af9d-ca9edd5c83da","Type":"ContainerStarted","Data":"6769174f8c703a056130a8d0740253e8b56ad60fee1a5767283d8d9d3b4bd1d1"}
Oct 30 19:13:58.285843 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:58.285784    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b" podStartSLOduration=2.285767925 podStartE2EDuration="2.285767925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:13:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:13:58.284929832 +0000 UTC m=+2358.996678533" watchObservedRunningTime="2026-10-30 19:13:58.285767925 +0000 UTC m=+2358.997516625"
Oct 30 19:13:58.945824 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:58.945791    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:58.951726 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:58.951702    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:59.099706 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:59.099682    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:59.100477 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:59.100457    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:13:59.749603 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:13:59.749570    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:01.120613 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:01.120572    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:01.374838 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:01.374753    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:02.596883 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:02.596821    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:05.098131 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:05.098093    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:05.760343 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:05.760307    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:06.675013 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:06.674975    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:07.440976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.440935    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"]
Oct 30 19:14:07.445157 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.445136    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.516350 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.516326    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-91cb32-openshift-default-dockercfg-zgn4s\""
Oct 30 19:14:07.541868 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.541827    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-workload-socket\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.542039 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.541910    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17ba2a01-0007-4f37-b1a9-464910fd82e6-istiod-ca-cert\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.542039 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.542000    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-envoy\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.542153 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.542045    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-token\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.542153 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.542104    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-data\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.542153 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.542125    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-credential-socket\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.542316 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.542257    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gznz\" (UniqueName: \"kubernetes.io/projected/17ba2a01-0007-4f37-b1a9-464910fd82e6-kube-api-access-2gznz\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.542316 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.542309    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-workload-certs\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.542485 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.542371    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-podinfo\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.560471 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.560446    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"]
Oct 30 19:14:07.585708 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.585667    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:07.643343 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.643307    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gznz\" (UniqueName: \"kubernetes.io/projected/17ba2a01-0007-4f37-b1a9-464910fd82e6-kube-api-access-2gznz\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.643534 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.643365    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-workload-certs\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.643534 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.643413    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-podinfo\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.643534 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.643443    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-workload-socket\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.643534 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.643468    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17ba2a01-0007-4f37-b1a9-464910fd82e6-istiod-ca-cert\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.643534 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.643512    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-envoy\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.643811 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.643538    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-token\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.643811 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.643611    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-data\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.643811 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.643642    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-credential-socket\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.644032 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.643884    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-workload-certs\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.644032 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.643971    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-workload-socket\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.644154 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.644134    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-credential-socket\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.644229 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.644156    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-data\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.644474 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.644444    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17ba2a01-0007-4f37-b1a9-464910fd82e6-istiod-ca-cert\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.646351 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.646326    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-envoy\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.646787 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.646768    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-podinfo\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.705554 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.705438    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-token\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.705919 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.705548    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gznz\" (UniqueName: \"kubernetes.io/projected/17ba2a01-0007-4f37-b1a9-464910fd82e6-kube-api-access-2gznz\") pod \"e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") " pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.759756 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.759723    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:07.809375 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:07.809341    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:08.085830 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:08.085795    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:08.199875 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:08.199821    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:14:08.200046 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:08.199971    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:14:08.200125 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:08.200071    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:14:08.201130 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:08.201105    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"]
Oct 30 19:14:08.224186 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:14:08.224158    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:14:08.279064 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:08.279035    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:09.172592 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:09.172556    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf" event={"ID":"17ba2a01-0007-4f37-b1a9-464910fd82e6","Type":"ContainerStarted","Data":"b38c1203a7f746311664f4a5481d28f31e69bb32071a0d451a1670d375f31cd8"}
Oct 30 19:14:09.173053 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:09.172612    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf" event={"ID":"17ba2a01-0007-4f37-b1a9-464910fd82e6","Type":"ContainerStarted","Data":"ee1360cb715a95231c44d5aa78a9803448cdc239c47bb54ffdb081358107efcd"}
Oct 30 19:14:09.216377 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:09.216350    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h"]
Oct 30 19:14:09.345910 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:09.345874    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72"]
Oct 30 19:14:09.346085 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:09.345992    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf" podStartSLOduration=2.345977584 podStartE2EDuration="2.345977584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:09.342318665 +0000 UTC m=+2370.054067386" watchObservedRunningTime="2026-10-30 19:14:09.345977584 +0000 UTC m=+2370.057726272"
Oct 30 19:14:09.634899 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:09.634861    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:09.760699 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:09.760662    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:09.766286 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:09.766260    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:09.875593 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:09.875555    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:10.179411 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:10.179375    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:10.180280 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:10.180262    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:14:13.619423 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:13.619386    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:13.809639 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:13.809605    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:13.993247 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:13.993175    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:14.217339 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:14.217297    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h" podUID="4923d0a0-12b1-4d37-b17c-bc7369ba6dad" containerName="payload-processing" containerID="cri-o://203b782717308cf22250645ad4e023a4afecad9d2b325431bcd9777e7c911101" gracePeriod=25
Oct 30 19:14:14.347099 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:14.347061    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72" podUID="c85bce08-7b30-4c2f-bc48-36f60dfad8d2" containerName="payload-pre-processing" containerID="cri-o://a3289f317c7f8595a5569e0210a1fc9f100ea2c994cda57a6a7878ed6017b1db" gracePeriod=25
Oct 30 19:14:14.510304 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:14.510273    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h"
Oct 30 19:14:14.609024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:14.608996    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4923d0a0-12b1-4d37-b17c-bc7369ba6dad-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4923d0a0-12b1-4d37-b17c-bc7369ba6dad-plugins-config-volume\") pod \"4923d0a0-12b1-4d37-b17c-bc7369ba6dad\" (UID: \"4923d0a0-12b1-4d37-b17c-bc7369ba6dad\") "
Oct 30 19:14:14.609149 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:14.609053    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4923d0a0-12b1-4d37-b17c-bc7369ba6dad-kube-api-access-qnxfl\" (UniqueName: \"kubernetes.io/projected/4923d0a0-12b1-4d37-b17c-bc7369ba6dad-kube-api-access-qnxfl\") pod \"4923d0a0-12b1-4d37-b17c-bc7369ba6dad\" (UID: \"4923d0a0-12b1-4d37-b17c-bc7369ba6dad\") "
Oct 30 19:14:14.609337 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:14.609317    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4923d0a0-12b1-4d37-b17c-bc7369ba6dad-plugins-config-volume" pod "4923d0a0-12b1-4d37-b17c-bc7369ba6dad" (UID: "4923d0a0-12b1-4d37-b17c-bc7369ba6dad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:14:14.609517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:14.609499    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4923d0a0-12b1-4d37-b17c-bc7369ba6dad-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:14:14.609832 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:14.609808    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72"
Oct 30 19:14:14.611237 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:14.611216    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4923d0a0-12b1-4d37-b17c-bc7369ba6dad-kube-api-access-qnxfl" pod "4923d0a0-12b1-4d37-b17c-bc7369ba6dad" (UID: "4923d0a0-12b1-4d37-b17c-bc7369ba6dad"). InnerVolumeSpecName "kube-api-access-qnxfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:14.710530 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:14.710499    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c85bce08-7b30-4c2f-bc48-36f60dfad8d2-kube-api-access-dnttn\" (UniqueName: \"kubernetes.io/projected/c85bce08-7b30-4c2f-bc48-36f60dfad8d2-kube-api-access-dnttn\") pod \"c85bce08-7b30-4c2f-bc48-36f60dfad8d2\" (UID: \"c85bce08-7b30-4c2f-bc48-36f60dfad8d2\") "
Oct 30 19:14:14.711025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:14.710642    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c85bce08-7b30-4c2f-bc48-36f60dfad8d2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c85bce08-7b30-4c2f-bc48-36f60dfad8d2-plugins-config-volume\") pod \"c85bce08-7b30-4c2f-bc48-36f60dfad8d2\" (UID: \"c85bce08-7b30-4c2f-bc48-36f60dfad8d2\") "
Oct 30 19:14:14.711025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:14.711013    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnxfl\" (UniqueName: \"kubernetes.io/projected/4923d0a0-12b1-4d37-b17c-bc7369ba6dad-kube-api-access-qnxfl\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:14:14.711157 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:14.711097    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c85bce08-7b30-4c2f-bc48-36f60dfad8d2-plugins-config-volume" pod "c85bce08-7b30-4c2f-bc48-36f60dfad8d2" (UID: "c85bce08-7b30-4c2f-bc48-36f60dfad8d2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:14:14.712684 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:14.712664    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c85bce08-7b30-4c2f-bc48-36f60dfad8d2-kube-api-access-dnttn" pod "c85bce08-7b30-4c2f-bc48-36f60dfad8d2" (UID: "c85bce08-7b30-4c2f-bc48-36f60dfad8d2"). InnerVolumeSpecName "kube-api-access-dnttn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:14.812517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:14.812467    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c85bce08-7b30-4c2f-bc48-36f60dfad8d2-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:14:14.812517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:14.812509    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnttn\" (UniqueName: \"kubernetes.io/projected/c85bce08-7b30-4c2f-bc48-36f60dfad8d2-kube-api-access-dnttn\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:14:14.944187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:14.944100    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:15.210190 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:15.210110    5820 generic.go:356] "Generic (PLEG): container finished" podID="4923d0a0-12b1-4d37-b17c-bc7369ba6dad" containerID="203b782717308cf22250645ad4e023a4afecad9d2b325431bcd9777e7c911101" exitCode=0
Oct 30 19:14:15.210190 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:15.210153    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h"
Oct 30 19:14:15.210444 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:15.210187    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h" event={"ID":"4923d0a0-12b1-4d37-b17c-bc7369ba6dad","Type":"ContainerDied","Data":"203b782717308cf22250645ad4e023a4afecad9d2b325431bcd9777e7c911101"}
Oct 30 19:14:15.210444 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:15.210257    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h" event={"ID":"4923d0a0-12b1-4d37-b17c-bc7369ba6dad","Type":"ContainerDied","Data":"b5753f942bfa7dca9f086c325035c56b20fe5b4619fd9a41e045314083be41b4"}
Oct 30 19:14:15.210444 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:15.210279    5820 scope.go:122] "RemoveContainer" containerID="203b782717308cf22250645ad4e023a4afecad9d2b325431bcd9777e7c911101"
Oct 30 19:14:15.211730 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:15.211703    5820 generic.go:356] "Generic (PLEG): container finished" podID="c85bce08-7b30-4c2f-bc48-36f60dfad8d2" containerID="a3289f317c7f8595a5569e0210a1fc9f100ea2c994cda57a6a7878ed6017b1db" exitCode=0
Oct 30 19:14:15.211876 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:15.211755    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72"
Oct 30 19:14:15.211876 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:15.211849    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72" event={"ID":"c85bce08-7b30-4c2f-bc48-36f60dfad8d2","Type":"ContainerDied","Data":"a3289f317c7f8595a5569e0210a1fc9f100ea2c994cda57a6a7878ed6017b1db"}
Oct 30 19:14:15.212041 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:15.211886    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72" event={"ID":"c85bce08-7b30-4c2f-bc48-36f60dfad8d2","Type":"ContainerDied","Data":"f30927d1e4bae7713c8954a12a426a638cc27bcc9d30e5026f45d8ba94309846"}
Oct 30 19:14:15.221368 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:15.221203    5820 scope.go:122] "RemoveContainer" containerID="203b782717308cf22250645ad4e023a4afecad9d2b325431bcd9777e7c911101"
Oct 30 19:14:15.221544 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:14:15.221521    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"203b782717308cf22250645ad4e023a4afecad9d2b325431bcd9777e7c911101\": container with ID starting with 203b782717308cf22250645ad4e023a4afecad9d2b325431bcd9777e7c911101 not found: ID does not exist" containerID="203b782717308cf22250645ad4e023a4afecad9d2b325431bcd9777e7c911101"
Oct 30 19:14:15.221609 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:15.221560    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"203b782717308cf22250645ad4e023a4afecad9d2b325431bcd9777e7c911101"} err="failed to get container status \"203b782717308cf22250645ad4e023a4afecad9d2b325431bcd9777e7c911101\": rpc error: code = NotFound desc = could not find container \"203b782717308cf22250645ad4e023a4afecad9d2b325431bcd9777e7c911101\": container with ID starting with 203b782717308cf22250645ad4e023a4afecad9d2b325431bcd9777e7c911101 not found: ID does not exist"
Oct 30 19:14:15.221609 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:15.221584    5820 scope.go:122] "RemoveContainer" containerID="a3289f317c7f8595a5569e0210a1fc9f100ea2c994cda57a6a7878ed6017b1db"
Oct 30 19:14:15.240058 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:15.240043    5820 scope.go:122] "RemoveContainer" containerID="a3289f317c7f8595a5569e0210a1fc9f100ea2c994cda57a6a7878ed6017b1db"
Oct 30 19:14:15.240279 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:14:15.240259    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3289f317c7f8595a5569e0210a1fc9f100ea2c994cda57a6a7878ed6017b1db\": container with ID starting with a3289f317c7f8595a5569e0210a1fc9f100ea2c994cda57a6a7878ed6017b1db not found: ID does not exist" containerID="a3289f317c7f8595a5569e0210a1fc9f100ea2c994cda57a6a7878ed6017b1db"
Oct 30 19:14:15.240362 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:15.240286    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3289f317c7f8595a5569e0210a1fc9f100ea2c994cda57a6a7878ed6017b1db"} err="failed to get container status \"a3289f317c7f8595a5569e0210a1fc9f100ea2c994cda57a6a7878ed6017b1db\": rpc error: code = NotFound desc = could not find container \"a3289f317c7f8595a5569e0210a1fc9f100ea2c994cda57a6a7878ed6017b1db\": container with ID starting with a3289f317c7f8595a5569e0210a1fc9f100ea2c994cda57a6a7878ed6017b1db not found: ID does not exist"
Oct 30 19:14:15.390021 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:15.389985    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72"]
Oct 30 19:14:15.399410 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:15.399385    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-54dc456fdc-g9j72"]
Oct 30 19:14:15.436833 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:15.436804    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c85bce08-7b30-4c2f-bc48-36f60dfad8d2" path="/var/lib/kubelet/pods/c85bce08-7b30-4c2f-bc48-36f60dfad8d2/volumes"
Oct 30 19:14:15.516344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:15.516277    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h"]
Oct 30 19:14:15.520098 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:15.520076    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-52d383-6c7fc5798-ql59h"]
Oct 30 19:14:17.005326 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.005273    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:17.237627 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.237587    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:17.437223 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.437193    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4923d0a0-12b1-4d37-b17c-bc7369ba6dad" path="/var/lib/kubelet/pods/4923d0a0-12b1-4d37-b17c-bc7369ba6dad/volumes"
Oct 30 19:14:17.484548 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.484514    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf"]
Oct 30 19:14:17.485179 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.485163    5820 cpu_manager.go:414] "containerMap: removing container" podUID="4923d0a0-12b1-4d37-b17c-bc7369ba6dad" containerName="payload-processing"
Oct 30 19:14:17.485248 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.485180    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4923d0a0-12b1-4d37-b17c-bc7369ba6dad" containerName="payload-processing"
Oct 30 19:14:17.485248 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.485216    5820 cpu_manager.go:414] "containerMap: removing container" podUID="c85bce08-7b30-4c2f-bc48-36f60dfad8d2" containerName="payload-pre-processing"
Oct 30 19:14:17.485248 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.485225    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c85bce08-7b30-4c2f-bc48-36f60dfad8d2" containerName="payload-pre-processing"
Oct 30 19:14:17.485350 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.485339    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="4923d0a0-12b1-4d37-b17c-bc7369ba6dad" containerName="payload-processing"
Oct 30 19:14:17.485384 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.485353    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="c85bce08-7b30-4c2f-bc48-36f60dfad8d2" containerName="payload-pre-processing"
Oct 30 19:14:17.490392 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.490373    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf"
Oct 30 19:14:17.566887 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.563815    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f04590de-490c-4c27-a42a-b17378b00e84-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf\" (UID: \"f04590de-490c-4c27-a42a-b17378b00e84\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf"
Oct 30 19:14:17.566887 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.564056    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqllb\" (UniqueName: \"kubernetes.io/projected/f04590de-490c-4c27-a42a-b17378b00e84-kube-api-access-lqllb\") pod \"payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf\" (UID: \"f04590de-490c-4c27-a42a-b17378b00e84\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf"
Oct 30 19:14:17.575745 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.571846    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf"]
Oct 30 19:14:17.613149 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.613030    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-91cb32\""
Oct 30 19:14:17.613149 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.613128    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-91cb32-dockercfg-fpfzr\""
Oct 30 19:14:17.665377 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.665349    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqllb\" (UniqueName: \"kubernetes.io/projected/f04590de-490c-4c27-a42a-b17378b00e84-kube-api-access-lqllb\") pod \"payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf\" (UID: \"f04590de-490c-4c27-a42a-b17378b00e84\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf"
Oct 30 19:14:17.665584 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.665414    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f04590de-490c-4c27-a42a-b17378b00e84-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf\" (UID: \"f04590de-490c-4c27-a42a-b17378b00e84\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf"
Oct 30 19:14:17.666158 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.666136    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f04590de-490c-4c27-a42a-b17378b00e84-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf\" (UID: \"f04590de-490c-4c27-a42a-b17378b00e84\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf"
Oct 30 19:14:17.713730 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.713662    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6"]
Oct 30 19:14:17.717739 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.717725    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6"
Oct 30 19:14:17.735030 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.735001    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqllb\" (UniqueName: \"kubernetes.io/projected/f04590de-490c-4c27-a42a-b17378b00e84-kube-api-access-lqllb\") pod \"payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf\" (UID: \"f04590de-490c-4c27-a42a-b17378b00e84\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf"
Oct 30 19:14:17.776181 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.776155    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6"]
Oct 30 19:14:17.804208 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.804178    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf"
Oct 30 19:14:17.871836 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.869363    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j59d\" (UniqueName: \"kubernetes.io/projected/bdbab4fc-60b7-4179-a386-7a8f96814802-kube-api-access-8j59d\") pod \"payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6\" (UID: \"bdbab4fc-60b7-4179-a386-7a8f96814802\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6"
Oct 30 19:14:17.871836 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.869414    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdbab4fc-60b7-4179-a386-7a8f96814802-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6\" (UID: \"bdbab4fc-60b7-4179-a386-7a8f96814802\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6"
Oct 30 19:14:17.970613 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.970535    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j59d\" (UniqueName: \"kubernetes.io/projected/bdbab4fc-60b7-4179-a386-7a8f96814802-kube-api-access-8j59d\") pod \"payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6\" (UID: \"bdbab4fc-60b7-4179-a386-7a8f96814802\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6"
Oct 30 19:14:17.970613 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.970571    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdbab4fc-60b7-4179-a386-7a8f96814802-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6\" (UID: \"bdbab4fc-60b7-4179-a386-7a8f96814802\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6"
Oct 30 19:14:17.971359 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:17.971330    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdbab4fc-60b7-4179-a386-7a8f96814802-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6\" (UID: \"bdbab4fc-60b7-4179-a386-7a8f96814802\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6"
Oct 30 19:14:18.032123 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:18.032090    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j59d\" (UniqueName: \"kubernetes.io/projected/bdbab4fc-60b7-4179-a386-7a8f96814802-kube-api-access-8j59d\") pod \"payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6\" (UID: \"bdbab4fc-60b7-4179-a386-7a8f96814802\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6"
Oct 30 19:14:18.181134 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:18.181110    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf"]
Oct 30 19:14:18.183237 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:14:18.183211    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf04590de_490c_4c27_a42a_b17378b00e84.slice/crio-249f554b2d91b1c000fc0c73a451e3a624ca01f5b3d735d3d1dda27ac9c3660b WatchSource:0}: Error finding container 249f554b2d91b1c000fc0c73a451e3a624ca01f5b3d735d3d1dda27ac9c3660b: Status 404 returned error can't find the container with id 249f554b2d91b1c000fc0c73a451e3a624ca01f5b3d735d3d1dda27ac9c3660b
Oct 30 19:14:18.235735 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:18.234066    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf" event={"ID":"f04590de-490c-4c27-a42a-b17378b00e84","Type":"ContainerStarted","Data":"249f554b2d91b1c000fc0c73a451e3a624ca01f5b3d735d3d1dda27ac9c3660b"}
Oct 30 19:14:18.331534 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:18.331506    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6"
Oct 30 19:14:18.709194 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:18.709166    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6"]
Oct 30 19:14:18.710749 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:14:18.710719    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdbab4fc_60b7_4179_a386_7a8f96814802.slice/crio-4414454dda51f743e44049906cab6de2683649c63709992e052e183a2971b200 WatchSource:0}: Error finding container 4414454dda51f743e44049906cab6de2683649c63709992e052e183a2971b200: Status 404 returned error can't find the container with id 4414454dda51f743e44049906cab6de2683649c63709992e052e183a2971b200
Oct 30 19:14:19.242291 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:19.242252    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf" event={"ID":"f04590de-490c-4c27-a42a-b17378b00e84","Type":"ContainerStarted","Data":"e8a06fef9cc531ded714873f9c16b02199c9abc0f822dbca596cc9bfa97c556e"}
Oct 30 19:14:19.242808 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:19.242316    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf"
Oct 30 19:14:19.243872 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:19.243844    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6" event={"ID":"bdbab4fc-60b7-4179-a386-7a8f96814802","Type":"ContainerStarted","Data":"e183d3abbee812c489abacd0c5f6dff711772f80fccfaab5c25fe03b31ecb1ff"}
Oct 30 19:14:19.244017 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:19.243895    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6" event={"ID":"bdbab4fc-60b7-4179-a386-7a8f96814802","Type":"ContainerStarted","Data":"4414454dda51f743e44049906cab6de2683649c63709992e052e183a2971b200"}
Oct 30 19:14:19.244017 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:19.243916    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6"
Oct 30 19:14:19.361745 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:19.361687    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf" podStartSLOduration=2.3616741660000002 podStartE2EDuration="2.361674166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:19.358896009 +0000 UTC m=+2380.070644706" watchObservedRunningTime="2026-10-30 19:14:19.361674166 +0000 UTC m=+2380.073422863"
Oct 30 19:14:19.477878 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:19.477827    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6" podStartSLOduration=2.477812358 podStartE2EDuration="2.477812358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:19.474488818 +0000 UTC m=+2380.186237515" watchObservedRunningTime="2026-10-30 19:14:19.477812358 +0000 UTC m=+2380.189561054"
Oct 30 19:14:19.606986 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:19.606926    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:21.262241 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:21.262200    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:21.507036 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:21.506997    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:21.689842 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:21.689796    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:22.200824 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:22.200786    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:27.065006 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:27.064937    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:29.027075 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:29.027038    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:29.204061 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:29.204023    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:29.386277 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:29.386241    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:29.571849 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:29.571815    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:30.252229 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:30.252200    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6"
Oct 30 19:14:30.252636 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:30.252518    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf"
Oct 30 19:14:31.499456 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:31.499421    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:32.259766 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:32.259725    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:37.520809 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:37.520773    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:38.203491 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:14:38.203458    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:14:42.685804 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:42.685770    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:46.038226 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:46.038190    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr"]
Oct 30 19:14:46.336658 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:46.336619    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:46.567549 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:46.567510    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:47.751273 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:47.751241    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:51.039120 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:51.039067    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr" podUID="5b9f46cd-813e-40f5-b4ec-6bea1b5bcead" containerName="payload-pre-processing" containerID="cri-o://54e1855315f3baae3482fc89118d1d46e0029841ca5887cca4a5c3f1b4456c9f" gracePeriod=25
Oct 30 19:14:51.308691 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:51.308656    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr"
Oct 30 19:14:51.334010 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:51.333980    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5b9f46cd-813e-40f5-b4ec-6bea1b5bcead-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b9f46cd-813e-40f5-b4ec-6bea1b5bcead-plugins-config-volume\") pod \"5b9f46cd-813e-40f5-b4ec-6bea1b5bcead\" (UID: \"5b9f46cd-813e-40f5-b4ec-6bea1b5bcead\") "
Oct 30 19:14:51.334175 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:51.334086    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b9f46cd-813e-40f5-b4ec-6bea1b5bcead-kube-api-access-9g6rr\" (UniqueName: \"kubernetes.io/projected/5b9f46cd-813e-40f5-b4ec-6bea1b5bcead-kube-api-access-9g6rr\") pod \"5b9f46cd-813e-40f5-b4ec-6bea1b5bcead\" (UID: \"5b9f46cd-813e-40f5-b4ec-6bea1b5bcead\") "
Oct 30 19:14:51.334327 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:51.334307    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b9f46cd-813e-40f5-b4ec-6bea1b5bcead-plugins-config-volume" pod "5b9f46cd-813e-40f5-b4ec-6bea1b5bcead" (UID: "5b9f46cd-813e-40f5-b4ec-6bea1b5bcead"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:14:51.334572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:51.334548    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b9f46cd-813e-40f5-b4ec-6bea1b5bcead-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:14:51.336346 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:51.336324    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b9f46cd-813e-40f5-b4ec-6bea1b5bcead-kube-api-access-9g6rr" pod "5b9f46cd-813e-40f5-b4ec-6bea1b5bcead" (UID: "5b9f46cd-813e-40f5-b4ec-6bea1b5bcead"). InnerVolumeSpecName "kube-api-access-9g6rr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:51.435547 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:51.435512    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9g6rr\" (UniqueName: \"kubernetes.io/projected/5b9f46cd-813e-40f5-b4ec-6bea1b5bcead-kube-api-access-9g6rr\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:14:51.537708 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:51.537674    5820 generic.go:356] "Generic (PLEG): container finished" podID="5b9f46cd-813e-40f5-b4ec-6bea1b5bcead" containerID="54e1855315f3baae3482fc89118d1d46e0029841ca5887cca4a5c3f1b4456c9f" exitCode=0
Oct 30 19:14:51.537855 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:51.537724    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr"
Oct 30 19:14:51.537855 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:51.537756    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr" event={"ID":"5b9f46cd-813e-40f5-b4ec-6bea1b5bcead","Type":"ContainerDied","Data":"54e1855315f3baae3482fc89118d1d46e0029841ca5887cca4a5c3f1b4456c9f"}
Oct 30 19:14:51.537855 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:51.537806    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr" event={"ID":"5b9f46cd-813e-40f5-b4ec-6bea1b5bcead","Type":"ContainerDied","Data":"a09f3c9268125ba0103eb87dd89a034d91552f4c54fe18b44accd6541cbf88c8"}
Oct 30 19:14:51.537855 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:51.537828    5820 scope.go:122] "RemoveContainer" containerID="54e1855315f3baae3482fc89118d1d46e0029841ca5887cca4a5c3f1b4456c9f"
Oct 30 19:14:51.547775 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:51.547760    5820 scope.go:122] "RemoveContainer" containerID="54e1855315f3baae3482fc89118d1d46e0029841ca5887cca4a5c3f1b4456c9f"
Oct 30 19:14:51.548100 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:14:51.548081    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54e1855315f3baae3482fc89118d1d46e0029841ca5887cca4a5c3f1b4456c9f\": container with ID starting with 54e1855315f3baae3482fc89118d1d46e0029841ca5887cca4a5c3f1b4456c9f not found: ID does not exist" containerID="54e1855315f3baae3482fc89118d1d46e0029841ca5887cca4a5c3f1b4456c9f"
Oct 30 19:14:51.548153 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:51.548110    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54e1855315f3baae3482fc89118d1d46e0029841ca5887cca4a5c3f1b4456c9f"} err="failed to get container status \"54e1855315f3baae3482fc89118d1d46e0029841ca5887cca4a5c3f1b4456c9f\": rpc error: code = NotFound desc = could not find container \"54e1855315f3baae3482fc89118d1d46e0029841ca5887cca4a5c3f1b4456c9f\": container with ID starting with 54e1855315f3baae3482fc89118d1d46e0029841ca5887cca4a5c3f1b4456c9f not found: ID does not exist"
Oct 30 19:14:51.704885 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:51.704797    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr"]
Oct 30 19:14:51.707274 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:51.707248    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-0be4a4-5bf9958887-rbznr"]
Oct 30 19:14:52.635753 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:52.634718    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:53.437035 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:53.436998    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b9f46cd-813e-40f5-b4ec-6bea1b5bcead" path="/var/lib/kubelet/pods/5b9f46cd-813e-40f5-b4ec-6bea1b5bcead/volumes"
Oct 30 19:14:57.748066 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:14:57.748030    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:00.086792 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:00.086739    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:00.222780 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:00.222736    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f"]
Oct 30 19:15:00.223407 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:00.223388    5820 cpu_manager.go:414] "containerMap: removing container" podUID="5b9f46cd-813e-40f5-b4ec-6bea1b5bcead" containerName="payload-pre-processing"
Oct 30 19:15:00.223538 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:00.223409    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b9f46cd-813e-40f5-b4ec-6bea1b5bcead" containerName="payload-pre-processing"
Oct 30 19:15:00.223618 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:00.223604    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b9f46cd-813e-40f5-b4ec-6bea1b5bcead" containerName="payload-pre-processing"
Oct 30 19:15:00.228273 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:00.228252    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f"
Oct 30 19:15:00.322248 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:00.322217    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qg7n\""
Oct 30 19:15:00.336774 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:00.336743    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f"]
Oct 30 19:15:00.349802 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:00.349725    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec95c24d-20b2-47bb-b621-418173534535-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f\" (UID: \"ec95c24d-20b2-47bb-b621-418173534535\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f"
Oct 30 19:15:00.349938 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:00.349895    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfzqx\" (UniqueName: \"kubernetes.io/projected/ec95c24d-20b2-47bb-b621-418173534535-kube-api-access-wfzqx\") pod \"maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f\" (UID: \"ec95c24d-20b2-47bb-b621-418173534535\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f"
Oct 30 19:15:00.451307 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:00.451269    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec95c24d-20b2-47bb-b621-418173534535-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f\" (UID: \"ec95c24d-20b2-47bb-b621-418173534535\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f"
Oct 30 19:15:00.451494 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:00.451393    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfzqx\" (UniqueName: \"kubernetes.io/projected/ec95c24d-20b2-47bb-b621-418173534535-kube-api-access-wfzqx\") pod \"maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f\" (UID: \"ec95c24d-20b2-47bb-b621-418173534535\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f"
Oct 30 19:15:00.451882 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:00.451862    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec95c24d-20b2-47bb-b621-418173534535-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f\" (UID: \"ec95c24d-20b2-47bb-b621-418173534535\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f"
Oct 30 19:15:00.517166 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:00.517135    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfzqx\" (UniqueName: \"kubernetes.io/projected/ec95c24d-20b2-47bb-b621-418173534535-kube-api-access-wfzqx\") pod \"maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f\" (UID: \"ec95c24d-20b2-47bb-b621-418173534535\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f"
Oct 30 19:15:00.542032 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:00.542001    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f"
Oct 30 19:15:00.944403 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:00.944371    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f"]
Oct 30 19:15:00.946189 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:15:00.946163    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec95c24d_20b2_47bb_b621_418173534535.slice/crio-c42c42c7ef3a6571cfa3b15ebc75757359d9284fe80f07fc1a5c921b440381af WatchSource:0}: Error finding container c42c42c7ef3a6571cfa3b15ebc75757359d9284fe80f07fc1a5c921b440381af: Status 404 returned error can't find the container with id c42c42c7ef3a6571cfa3b15ebc75757359d9284fe80f07fc1a5c921b440381af
Oct 30 19:15:01.301514 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:01.301478    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:01.608617 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:01.608573    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f" event={"ID":"ec95c24d-20b2-47bb-b621-418173534535","Type":"ContainerStarted","Data":"abe77c4dddd0359c6c5aeffa403aad40b931dbf3460025ee01e9dba74fd1f3b3"}
Oct 30 19:15:01.608839 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:01.608632    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f" event={"ID":"ec95c24d-20b2-47bb-b621-418173534535","Type":"ContainerStarted","Data":"c42c42c7ef3a6571cfa3b15ebc75757359d9284fe80f07fc1a5c921b440381af"}
Oct 30 19:15:01.725895 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:01.725840    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f" podStartSLOduration=1.725825485 podStartE2EDuration="1.725825485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:01.724215865 +0000 UTC m=+2422.435964563" watchObservedRunningTime="2026-10-30 19:15:01.725825485 +0000 UTC m=+2422.437574182"
Oct 30 19:15:01.805229 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:01.805092    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:02.616272 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:02.616231    5820 generic.go:356] "Generic (PLEG): container finished" podID="ec95c24d-20b2-47bb-b621-418173534535" containerID="abe77c4dddd0359c6c5aeffa403aad40b931dbf3460025ee01e9dba74fd1f3b3" exitCode=60
Oct 30 19:15:02.616690 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:02.616318    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f" event={"ID":"ec95c24d-20b2-47bb-b621-418173534535","Type":"ContainerDied","Data":"abe77c4dddd0359c6c5aeffa403aad40b931dbf3460025ee01e9dba74fd1f3b3"}
Oct 30 19:15:02.616690 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:02.616560    5820 scope.go:122] "RemoveContainer" containerID="abe77c4dddd0359c6c5aeffa403aad40b931dbf3460025ee01e9dba74fd1f3b3"
Oct 30 19:15:03.624619 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:03.624586    5820 generic.go:356] "Generic (PLEG): container finished" podID="ec95c24d-20b2-47bb-b621-418173534535" containerID="eb38a89ede14bb9df356aaba593a458ecc4714025b5c88b39a3730fd5b5f09c0" exitCode=60
Oct 30 19:15:03.625108 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:03.624630    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f" event={"ID":"ec95c24d-20b2-47bb-b621-418173534535","Type":"ContainerDied","Data":"eb38a89ede14bb9df356aaba593a458ecc4714025b5c88b39a3730fd5b5f09c0"}
Oct 30 19:15:03.625108 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:03.624675    5820 scope.go:122] "RemoveContainer" containerID="abe77c4dddd0359c6c5aeffa403aad40b931dbf3460025ee01e9dba74fd1f3b3"
Oct 30 19:15:03.625108 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:03.625010    5820 scope.go:122] "RemoveContainer" containerID="eb38a89ede14bb9df356aaba593a458ecc4714025b5c88b39a3730fd5b5f09c0"
Oct 30 19:15:03.625391 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:15:03.625359    5820 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f_odh-ai-gateway-infra(ec95c24d-20b2-47bb-b621-418173534535)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f" podUID="ec95c24d-20b2-47bb-b621-418173534535"
Oct 30 19:15:04.327791 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:04.327751    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"]
Oct 30 19:15:04.332743 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:04.332725    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"
Oct 30 19:15:04.387919 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:04.387882    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-6d96a6-serving-cert\""
Oct 30 19:15:04.391513 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:04.391481    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"]
Oct 30 19:15:04.407120 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:04.407087    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dd29180-7684-4009-9975-1be5c491d9a8-maas-api-tls\") pod \"maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx\" (UID: \"1dd29180-7684-4009-9975-1be5c491d9a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"
Oct 30 19:15:04.407268 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:04.407236    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h479\" (UniqueName: \"kubernetes.io/projected/1dd29180-7684-4009-9975-1be5c491d9a8-kube-api-access-7h479\") pod \"maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx\" (UID: \"1dd29180-7684-4009-9975-1be5c491d9a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"
Oct 30 19:15:04.407407 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:04.407389    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dd29180-7684-4009-9975-1be5c491d9a8-metrics-certs\") pod \"maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx\" (UID: \"1dd29180-7684-4009-9975-1be5c491d9a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"
Oct 30 19:15:04.407494 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:04.407478    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dd29180-7684-4009-9975-1be5c491d9a8-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx\" (UID: \"1dd29180-7684-4009-9975-1be5c491d9a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"
Oct 30 19:15:04.508413 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:04.508371    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dd29180-7684-4009-9975-1be5c491d9a8-metrics-certs\") pod \"maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx\" (UID: \"1dd29180-7684-4009-9975-1be5c491d9a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"
Oct 30 19:15:04.508607 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:04.508470    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dd29180-7684-4009-9975-1be5c491d9a8-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx\" (UID: \"1dd29180-7684-4009-9975-1be5c491d9a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"
Oct 30 19:15:04.508607 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:04.508550    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dd29180-7684-4009-9975-1be5c491d9a8-maas-api-tls\") pod \"maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx\" (UID: \"1dd29180-7684-4009-9975-1be5c491d9a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"
Oct 30 19:15:04.508725 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:04.508630    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h479\" (UniqueName: \"kubernetes.io/projected/1dd29180-7684-4009-9975-1be5c491d9a8-kube-api-access-7h479\") pod \"maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx\" (UID: \"1dd29180-7684-4009-9975-1be5c491d9a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"
Oct 30 19:15:04.508725 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:15:04.508702    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-serving-cert: secret "maas-api-e2e-praxis-b-6d96a6-serving-cert" not found
Oct 30 19:15:04.508806 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:15:04.508780    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dd29180-7684-4009-9975-1be5c491d9a8-maas-api-tls podName:1dd29180-7684-4009-9975-1be5c491d9a8 nodeName:}" failed. No retries permitted until 2026-10-30 19:15:05.008759355 +0000 UTC m=+2425.720508046 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1dd29180-7684-4009-9975-1be5c491d9a8-maas-api-tls") pod "maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx" (UID: "1dd29180-7684-4009-9975-1be5c491d9a8") : secret "maas-api-e2e-praxis-b-6d96a6-serving-cert" not found
Oct 30 19:15:04.509166 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:04.509142    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dd29180-7684-4009-9975-1be5c491d9a8-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx\" (UID: \"1dd29180-7684-4009-9975-1be5c491d9a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"
Oct 30 19:15:04.511096 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:04.511075    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dd29180-7684-4009-9975-1be5c491d9a8-metrics-certs\") pod \"maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx\" (UID: \"1dd29180-7684-4009-9975-1be5c491d9a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"
Oct 30 19:15:04.564131 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:04.564103    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:04.569709 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:04.569691    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h479\" (UniqueName: \"kubernetes.io/projected/1dd29180-7684-4009-9975-1be5c491d9a8-kube-api-access-7h479\") pod \"maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx\" (UID: \"1dd29180-7684-4009-9975-1be5c491d9a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"
Oct 30 19:15:04.633046 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:04.632979    5820 scope.go:122] "RemoveContainer" containerID="eb38a89ede14bb9df356aaba593a458ecc4714025b5c88b39a3730fd5b5f09c0"
Oct 30 19:15:04.633378 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:15:04.633236    5820 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f_odh-ai-gateway-infra(ec95c24d-20b2-47bb-b621-418173534535)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f" podUID="ec95c24d-20b2-47bb-b621-418173534535"
Oct 30 19:15:04.745477 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:04.745441    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:05.014488 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:05.014398    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dd29180-7684-4009-9975-1be5c491d9a8-maas-api-tls\") pod \"maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx\" (UID: \"1dd29180-7684-4009-9975-1be5c491d9a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"
Oct 30 19:15:05.017042 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:05.017022    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dd29180-7684-4009-9975-1be5c491d9a8-maas-api-tls\") pod \"maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx\" (UID: \"1dd29180-7684-4009-9975-1be5c491d9a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"
Oct 30 19:15:05.246717 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:05.246687    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"
Oct 30 19:15:05.623646 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:05.623622    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"]
Oct 30 19:15:05.625603 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:15:05.625575    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dd29180_7684_4009_9975_1be5c491d9a8.slice/crio-696c8410df0e392ec641e7952f2ff3cf1ab0dbeb207a4d702bb5b63091b8d414 WatchSource:0}: Error finding container 696c8410df0e392ec641e7952f2ff3cf1ab0dbeb207a4d702bb5b63091b8d414: Status 404 returned error can't find the container with id 696c8410df0e392ec641e7952f2ff3cf1ab0dbeb207a4d702bb5b63091b8d414
Oct 30 19:15:05.640172 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:05.640132    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx" event={"ID":"1dd29180-7684-4009-9975-1be5c491d9a8","Type":"ContainerStarted","Data":"696c8410df0e392ec641e7952f2ff3cf1ab0dbeb207a4d702bb5b63091b8d414"}
Oct 30 19:15:06.654151 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:06.654106    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx" event={"ID":"1dd29180-7684-4009-9975-1be5c491d9a8","Type":"ContainerStarted","Data":"fb335bd6655853bd67ad0c2559d648faea6ec4b3b9c2511a24b2722db5a9538e"}
Oct 30 19:15:06.654606 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:06.654328    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"
Oct 30 19:15:06.780338 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:06.780245    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx" podStartSLOduration=1.920819496 podStartE2EDuration="2.780231109s" totalImagesPullingTime="859.411613ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:06.77779627 +0000 UTC m=+2427.489544967" watchObservedRunningTime="2026-10-30 19:15:06.780231109 +0000 UTC m=+2427.491979805"
Oct 30 19:15:08.204521 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:15:08.204478    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:15:09.758554 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:09.758282    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:12.667006 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:12.666936    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"
Oct 30 19:15:15.784415 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:15.784367    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"]
Oct 30 19:15:15.784936 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:15.784640    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds" podUID="8201f1d6-e2ee-493b-81f1-77081a4d4db8" containerName="maas-api" containerID="cri-o://924ff12d171faabfd5a1eede837ed31b00c11f51d78c09403fa7b331fce83d8d" gracePeriod=30
Oct 30 19:15:15.932563 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:15.932525    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:16.080714 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:16.080685    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"
Oct 30 19:15:16.199514 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:16.199472    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8201f1d6-e2ee-493b-81f1-77081a4d4db8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8201f1d6-e2ee-493b-81f1-77081a4d4db8-maas-api-tls\") pod \"8201f1d6-e2ee-493b-81f1-77081a4d4db8\" (UID: \"8201f1d6-e2ee-493b-81f1-77081a4d4db8\") "
Oct 30 19:15:16.201252 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:16.201227    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8201f1d6-e2ee-493b-81f1-77081a4d4db8-kube-api-access-8s2hp\" (UniqueName: \"kubernetes.io/projected/8201f1d6-e2ee-493b-81f1-77081a4d4db8-kube-api-access-8s2hp\") pod \"8201f1d6-e2ee-493b-81f1-77081a4d4db8\" (UID: \"8201f1d6-e2ee-493b-81f1-77081a4d4db8\") "
Oct 30 19:15:16.201766 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:16.201747    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8201f1d6-e2ee-493b-81f1-77081a4d4db8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8201f1d6-e2ee-493b-81f1-77081a4d4db8-metrics-certs\") pod \"8201f1d6-e2ee-493b-81f1-77081a4d4db8\" (UID: \"8201f1d6-e2ee-493b-81f1-77081a4d4db8\") "
Oct 30 19:15:16.202418 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:16.202398    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8201f1d6-e2ee-493b-81f1-77081a4d4db8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8201f1d6-e2ee-493b-81f1-77081a4d4db8-openshift-service-ca\") pod \"8201f1d6-e2ee-493b-81f1-77081a4d4db8\" (UID: \"8201f1d6-e2ee-493b-81f1-77081a4d4db8\") "
Oct 30 19:15:16.208934 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:16.208805    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8201f1d6-e2ee-493b-81f1-77081a4d4db8-openshift-service-ca" pod "8201f1d6-e2ee-493b-81f1-77081a4d4db8" (UID: "8201f1d6-e2ee-493b-81f1-77081a4d4db8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:16.209691 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:16.209667    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8201f1d6-e2ee-493b-81f1-77081a4d4db8-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:16.218222 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:16.218170    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8201f1d6-e2ee-493b-81f1-77081a4d4db8-kube-api-access-8s2hp" pod "8201f1d6-e2ee-493b-81f1-77081a4d4db8" (UID: "8201f1d6-e2ee-493b-81f1-77081a4d4db8"). InnerVolumeSpecName "kube-api-access-8s2hp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:16.219028 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:16.218981    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8201f1d6-e2ee-493b-81f1-77081a4d4db8-maas-api-tls" pod "8201f1d6-e2ee-493b-81f1-77081a4d4db8" (UID: "8201f1d6-e2ee-493b-81f1-77081a4d4db8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:16.219176 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:16.219098    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8201f1d6-e2ee-493b-81f1-77081a4d4db8-metrics-certs" pod "8201f1d6-e2ee-493b-81f1-77081a4d4db8" (UID: "8201f1d6-e2ee-493b-81f1-77081a4d4db8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:16.311080 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:16.311002    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8201f1d6-e2ee-493b-81f1-77081a4d4db8-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:16.311080 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:16.311030    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8s2hp\" (UniqueName: \"kubernetes.io/projected/8201f1d6-e2ee-493b-81f1-77081a4d4db8-kube-api-access-8s2hp\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:16.311080 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:16.311041    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8201f1d6-e2ee-493b-81f1-77081a4d4db8-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:16.719702 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:16.719612    5820 generic.go:356] "Generic (PLEG): container finished" podID="8201f1d6-e2ee-493b-81f1-77081a4d4db8" containerID="924ff12d171faabfd5a1eede837ed31b00c11f51d78c09403fa7b331fce83d8d" exitCode=0
Oct 30 19:15:16.719702 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:16.719663    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"
Oct 30 19:15:16.719976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:16.719702    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds" event={"ID":"8201f1d6-e2ee-493b-81f1-77081a4d4db8","Type":"ContainerDied","Data":"924ff12d171faabfd5a1eede837ed31b00c11f51d78c09403fa7b331fce83d8d"}
Oct 30 19:15:16.719976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:16.719771    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds" event={"ID":"8201f1d6-e2ee-493b-81f1-77081a4d4db8","Type":"ContainerDied","Data":"284c0725d674d724bc6bf742d7d19f7b760f1e6375bff0912f0b31e9e24a37ec"}
Oct 30 19:15:16.719976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:16.719799    5820 scope.go:122] "RemoveContainer" containerID="924ff12d171faabfd5a1eede837ed31b00c11f51d78c09403fa7b331fce83d8d"
Oct 30 19:15:16.731943 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:16.731923    5820 scope.go:122] "RemoveContainer" containerID="924ff12d171faabfd5a1eede837ed31b00c11f51d78c09403fa7b331fce83d8d"
Oct 30 19:15:16.732262 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:15:16.732239    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"924ff12d171faabfd5a1eede837ed31b00c11f51d78c09403fa7b331fce83d8d\": container with ID starting with 924ff12d171faabfd5a1eede837ed31b00c11f51d78c09403fa7b331fce83d8d not found: ID does not exist" containerID="924ff12d171faabfd5a1eede837ed31b00c11f51d78c09403fa7b331fce83d8d"
Oct 30 19:15:16.732332 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:16.732274    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"924ff12d171faabfd5a1eede837ed31b00c11f51d78c09403fa7b331fce83d8d"} err="failed to get container status \"924ff12d171faabfd5a1eede837ed31b00c11f51d78c09403fa7b331fce83d8d\": rpc error: code = NotFound desc = could not find container \"924ff12d171faabfd5a1eede837ed31b00c11f51d78c09403fa7b331fce83d8d\": container with ID starting with 924ff12d171faabfd5a1eede837ed31b00c11f51d78c09403fa7b331fce83d8d not found: ID does not exist"
Oct 30 19:15:16.891192 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:16.891158    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"]
Oct 30 19:15:16.903439 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:16.903411    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-0be4a4-585b78f576-4lfds"]
Oct 30 19:15:17.429406 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:17.429334    5820 scope.go:122] "RemoveContainer" containerID="eb38a89ede14bb9df356aaba593a458ecc4714025b5c88b39a3730fd5b5f09c0"
Oct 30 19:15:17.438336 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:17.438309    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8201f1d6-e2ee-493b-81f1-77081a4d4db8" path="/var/lib/kubelet/pods/8201f1d6-e2ee-493b-81f1-77081a4d4db8/volumes"
Oct 30 19:15:17.463176 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:17.461877    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"]
Oct 30 19:15:17.463176 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:17.462767    5820 cpu_manager.go:414] "containerMap: removing container" podUID="8201f1d6-e2ee-493b-81f1-77081a4d4db8" containerName="maas-api"
Oct 30 19:15:17.463176 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:17.462786    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8201f1d6-e2ee-493b-81f1-77081a4d4db8" containerName="maas-api"
Oct 30 19:15:17.463176 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:17.462921    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="8201f1d6-e2ee-493b-81f1-77081a4d4db8" containerName="maas-api"
Oct 30 19:15:17.467760 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:17.467742    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"
Oct 30 19:15:17.527069 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:17.527041    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"]
Oct 30 19:15:17.533267 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:17.533235    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75wbm\" (UniqueName: \"kubernetes.io/projected/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-kube-api-access-75wbm\") pod \"maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg\" (UID: \"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"
Oct 30 19:15:17.533400 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:17.533362    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg\" (UID: \"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"
Oct 30 19:15:17.533494 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:17.533470    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-maas-api-tls\") pod \"maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg\" (UID: \"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"
Oct 30 19:15:17.533582 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:17.533570    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-metrics-certs\") pod \"maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg\" (UID: \"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"
Oct 30 19:15:17.534835 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:17.534793    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-91cb32-serving-cert\""
Oct 30 19:15:17.579944 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:17.579912    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:17.634854 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:17.634823    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-metrics-certs\") pod \"maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg\" (UID: \"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"
Oct 30 19:15:17.635037 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:17.634944    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75wbm\" (UniqueName: \"kubernetes.io/projected/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-kube-api-access-75wbm\") pod \"maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg\" (UID: \"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"
Oct 30 19:15:17.635089 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:17.635035    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg\" (UID: \"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"
Oct 30 19:15:17.635133 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:17.635104    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-maas-api-tls\") pod \"maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg\" (UID: \"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"
Oct 30 19:15:17.635266 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:15:17.635246    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-serving-cert: secret "maas-api-e2e-ipp-b-91cb32-serving-cert" not found
Oct 30 19:15:17.635344 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:15:17.635323    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-maas-api-tls podName:0bb5a3ba-012c-41e0-bc65-e12c59bc9e15 nodeName:}" failed. No retries permitted until 2026-10-30 19:15:18.135302124 +0000 UTC m=+2438.847050813 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-maas-api-tls") pod "maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg" (UID: "0bb5a3ba-012c-41e0-bc65-e12c59bc9e15") : secret "maas-api-e2e-ipp-b-91cb32-serving-cert" not found
Oct 30 19:15:17.635645 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:17.635625    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg\" (UID: \"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"
Oct 30 19:15:17.637682 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:17.637660    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-metrics-certs\") pod \"maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg\" (UID: \"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"
Oct 30 19:15:17.695774 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:17.695710    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75wbm\" (UniqueName: \"kubernetes.io/projected/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-kube-api-access-75wbm\") pod \"maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg\" (UID: \"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"
Oct 30 19:15:17.822761 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:17.822712    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:18.004513 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:18.004407    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:18.146568 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:18.146225    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-maas-api-tls\") pod \"maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg\" (UID: \"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"
Oct 30 19:15:18.149436 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:18.149404    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-maas-api-tls\") pod \"maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg\" (UID: \"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"
Oct 30 19:15:18.182272 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:18.182235    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:18.387782 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:18.387753    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"
Oct 30 19:15:18.472883 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:18.472803    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:18.752262 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:18.752183    5820 generic.go:356] "Generic (PLEG): container finished" podID="ec95c24d-20b2-47bb-b621-418173534535" containerID="caa009664dd5c1f7cf2fe2635a400090cedfd62d5015c1906a14d5f4186abed2" exitCode=60
Oct 30 19:15:18.752445 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:18.752258    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f" event={"ID":"ec95c24d-20b2-47bb-b621-418173534535","Type":"ContainerDied","Data":"caa009664dd5c1f7cf2fe2635a400090cedfd62d5015c1906a14d5f4186abed2"}
Oct 30 19:15:18.752445 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:18.752332    5820 scope.go:122] "RemoveContainer" containerID="eb38a89ede14bb9df356aaba593a458ecc4714025b5c88b39a3730fd5b5f09c0"
Oct 30 19:15:18.752782 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:18.752757    5820 scope.go:122] "RemoveContainer" containerID="caa009664dd5c1f7cf2fe2635a400090cedfd62d5015c1906a14d5f4186abed2"
Oct 30 19:15:18.753195 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:15:18.753169    5820 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f_odh-ai-gateway-infra(ec95c24d-20b2-47bb-b621-418173534535)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f" podUID="ec95c24d-20b2-47bb-b621-418173534535"
Oct 30 19:15:18.763030 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:18.763009    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"]
Oct 30 19:15:18.764809 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:15:18.764782    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bb5a3ba_012c_41e0_bc65_e12c59bc9e15.slice/crio-6df583f2ce6f8e58f6b1320468c58136a1b424d21f79d03b63c686d6c6cdb68b WatchSource:0}: Error finding container 6df583f2ce6f8e58f6b1320468c58136a1b424d21f79d03b63c686d6c6cdb68b: Status 404 returned error can't find the container with id 6df583f2ce6f8e58f6b1320468c58136a1b424d21f79d03b63c686d6c6cdb68b
Oct 30 19:15:19.762639 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:19.762606    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg" event={"ID":"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15","Type":"ContainerStarted","Data":"69e770d8e6b2c4e27e27bb86306cfb62f503a404b49b7d10df129d8db1c96613"}
Oct 30 19:15:19.763129 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:19.762655    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg" event={"ID":"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15","Type":"ContainerStarted","Data":"6df583f2ce6f8e58f6b1320468c58136a1b424d21f79d03b63c686d6c6cdb68b"}
Oct 30 19:15:19.763129 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:19.762720    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"
Oct 30 19:15:19.885014 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:19.884931    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f"]
Oct 30 19:15:19.932548 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:19.932489    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg" podStartSLOduration=2.070995669 podStartE2EDuration="2.932475059s" totalImagesPullingTime="861.47939ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:19.929224288 +0000 UTC m=+2440.640972984" watchObservedRunningTime="2026-10-30 19:15:19.932475059 +0000 UTC m=+2440.644223796"
Oct 30 19:15:20.080042 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:20.080013    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f"
Oct 30 19:15:20.178262 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:20.178179    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec95c24d-20b2-47bb-b621-418173534535-kube-api-access-wfzqx\" (UniqueName: \"kubernetes.io/projected/ec95c24d-20b2-47bb-b621-418173534535-kube-api-access-wfzqx\") pod \"ec95c24d-20b2-47bb-b621-418173534535\" (UID: \"ec95c24d-20b2-47bb-b621-418173534535\") "
Oct 30 19:15:20.178262 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:20.178229    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ec95c24d-20b2-47bb-b621-418173534535-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec95c24d-20b2-47bb-b621-418173534535-maas-api-service-ca\") pod \"ec95c24d-20b2-47bb-b621-418173534535\" (UID: \"ec95c24d-20b2-47bb-b621-418173534535\") "
Oct 30 19:15:20.178630 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:20.178607    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec95c24d-20b2-47bb-b621-418173534535-maas-api-service-ca" pod "ec95c24d-20b2-47bb-b621-418173534535" (UID: "ec95c24d-20b2-47bb-b621-418173534535"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:20.180527 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:20.180506    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec95c24d-20b2-47bb-b621-418173534535-kube-api-access-wfzqx" pod "ec95c24d-20b2-47bb-b621-418173534535" (UID: "ec95c24d-20b2-47bb-b621-418173534535"). InnerVolumeSpecName "kube-api-access-wfzqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:20.279604 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:20.279573    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfzqx\" (UniqueName: \"kubernetes.io/projected/ec95c24d-20b2-47bb-b621-418173534535-kube-api-access-wfzqx\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:20.279604 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:20.279602    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec95c24d-20b2-47bb-b621-418173534535-maas-api-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:20.771747 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:20.771717    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f"
Oct 30 19:15:20.772283 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:20.772229    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f" event={"ID":"ec95c24d-20b2-47bb-b621-418173534535","Type":"ContainerDied","Data":"c42c42c7ef3a6571cfa3b15ebc75757359d9284fe80f07fc1a5c921b440381af"}
Oct 30 19:15:20.772367 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:20.772292    5820 scope.go:122] "RemoveContainer" containerID="caa009664dd5c1f7cf2fe2635a400090cedfd62d5015c1906a14d5f4186abed2"
Oct 30 19:15:20.941756 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:20.941725    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f"]
Oct 30 19:15:20.946278 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:20.946256    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7f44c7-29889795-m7p8f"]
Oct 30 19:15:21.437194 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:21.437156    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ec95c24d-20b2-47bb-b621-418173534535" path="/var/lib/kubelet/pods/ec95c24d-20b2-47bb-b621-418173534535/volumes"
Oct 30 19:15:23.096344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.096308    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"]
Oct 30 19:15:23.097103 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.097084    5820 cpu_manager.go:414] "containerMap: removing container" podUID="ec95c24d-20b2-47bb-b621-418173534535" containerName="cleanup"
Oct 30 19:15:23.097103 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.097103    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec95c24d-20b2-47bb-b621-418173534535" containerName="cleanup"
Oct 30 19:15:23.097266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.097117    5820 cpu_manager.go:414] "containerMap: removing container" podUID="ec95c24d-20b2-47bb-b621-418173534535" containerName="cleanup"
Oct 30 19:15:23.097266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.097125    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec95c24d-20b2-47bb-b621-418173534535" containerName="cleanup"
Oct 30 19:15:23.097266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.097150    5820 cpu_manager.go:414] "containerMap: removing container" podUID="ec95c24d-20b2-47bb-b621-418173534535" containerName="cleanup"
Oct 30 19:15:23.097266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.097156    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec95c24d-20b2-47bb-b621-418173534535" containerName="cleanup"
Oct 30 19:15:23.097266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.097244    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec95c24d-20b2-47bb-b621-418173534535" containerName="cleanup"
Oct 30 19:15:23.097266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.097252    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec95c24d-20b2-47bb-b621-418173534535" containerName="cleanup"
Oct 30 19:15:23.097266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.097263    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec95c24d-20b2-47bb-b621-418173534535" containerName="cleanup"
Oct 30 19:15:23.101900 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.101882    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.138816 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.138767    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:23.158020 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.157767    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-fd61e3-openshift-default-dockercfg-tn6nh\""
Oct 30 19:15:23.210985 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.210939    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"]
Oct 30 19:15:23.213186 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.213158    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6v9f\" (UniqueName: \"kubernetes.io/projected/166098fd-c83d-4ce8-9b4c-9a69751e1e17-kube-api-access-d6v9f\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.213354 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.213192    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.213354 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.213233    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.213354 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.213339    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.213538 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.213360    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.213538 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.213380    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.213538 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.213414    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.213712 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.213592    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.213712 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.213617    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.314383 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.314345    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6v9f\" (UniqueName: \"kubernetes.io/projected/166098fd-c83d-4ce8-9b4c-9a69751e1e17-kube-api-access-d6v9f\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.314383 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.314385    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.314605 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.314431    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.314605 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.314505    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.314605 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.314527    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.314605 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.314555    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.314839 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.314609    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.314839 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.314701    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.314839 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.314738    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.315056 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.314895    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.315126 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.315079    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.315126 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.315110    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.315202 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.315147    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.315286 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.315265    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.317689 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.317667    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.318037 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.318016    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.375851 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.375769    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.376013 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.375985    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6v9f\" (UniqueName: \"kubernetes.io/projected/166098fd-c83d-4ce8-9b4c-9a69751e1e17-kube-api-access-d6v9f\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.415762 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.415720    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:23.854573 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.854540    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"]
Oct 30 19:15:23.855658 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:15:23.855622    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod166098fd_c83d_4ce8_9b4c_9a69751e1e17.slice/crio-be8d5127b6436a439ddb7458002641ff000a8284bdfa1c58f4530bfc8aa9d3aa WatchSource:0}: Error finding container be8d5127b6436a439ddb7458002641ff000a8284bdfa1c58f4530bfc8aa9d3aa: Status 404 returned error can't find the container with id be8d5127b6436a439ddb7458002641ff000a8284bdfa1c58f4530bfc8aa9d3aa
Oct 30 19:15:23.858336 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.858301    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:15:23.858432 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.858368    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:15:23.858432 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:23.858398    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:15:24.799028 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:24.798986    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d" event={"ID":"166098fd-c83d-4ce8-9b4c-9a69751e1e17","Type":"ContainerStarted","Data":"d044d9298c85a5548a0995a4a0cc9e11de550d33e5fcd0b83967f61657af9777"}
Oct 30 19:15:24.799460 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:24.799045    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d" event={"ID":"166098fd-c83d-4ce8-9b4c-9a69751e1e17","Type":"ContainerStarted","Data":"be8d5127b6436a439ddb7458002641ff000a8284bdfa1c58f4530bfc8aa9d3aa"}
Oct 30 19:15:24.931217 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:24.931156    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d" podStartSLOduration=1.931141386 podStartE2EDuration="1.931141386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:24.930203375 +0000 UTC m=+2445.641952074" watchObservedRunningTime="2026-10-30 19:15:24.931141386 +0000 UTC m=+2445.642890082"
Oct 30 19:15:25.105300 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:25.105175    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:25.359036 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:25.356546    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:25.416137 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:25.416107    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:25.417744 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:25.417703    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Oct 30 19:15:25.417891 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:25.417770    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d" podUID="166098fd-c83d-4ce8-9b4c-9a69751e1e17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Oct 30 19:15:25.471482 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:25.471450    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"]
Oct 30 19:15:25.471673 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:25.471651    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6" podUID="4a0996ab-5aaa-4b71-b064-87d9d1572d13" containerName="istio-proxy" containerID="cri-o://eeafa7b7322daa533729b4543a212ac1f6cc19b8ff9cf0832fd11353f8675506" gracePeriod=30
Oct 30 19:15:25.559029 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:25.557825    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:25.779180 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:25.779101    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"
Oct 30 19:15:26.416498 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:26.416465    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Oct 30 19:15:26.416875 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:26.416517    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d" podUID="166098fd-c83d-4ce8-9b4c-9a69751e1e17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Oct 30 19:15:27.416268 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:27.416231    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Oct 30 19:15:27.416457 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:27.416289    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d" podUID="166098fd-c83d-4ce8-9b4c-9a69751e1e17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Oct 30 19:15:28.123246 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:28.123206    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:28.416431 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:28.416343    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Oct 30 19:15:28.416431 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:28.416400    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d" podUID="166098fd-c83d-4ce8-9b4c-9a69751e1e17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Oct 30 19:15:28.459615 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:28.459569    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:28.656747 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:28.656711    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:29.059246 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:29.059201    5820 patch_prober.go:29] interesting pod/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:15:29.059435 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:29.059277    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6" podUID="4a0996ab-5aaa-4b71-b064-87d9d1572d13" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:15:29.416240 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:29.416155    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Oct 30 19:15:29.416240 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:29.416211    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d" podUID="166098fd-c83d-4ce8-9b4c-9a69751e1e17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Oct 30 19:15:30.417035 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.416995    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Oct 30 19:15:30.417469 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.417053    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d" podUID="166098fd-c83d-4ce8-9b4c-9a69751e1e17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Oct 30 19:15:30.739065 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.739041    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:15:30.834155 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.834109    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-envoy\") pod \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") "
Oct 30 19:15:30.834326 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.834200    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-token\" (UniqueName: \"kubernetes.io/projected/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-token\") pod \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") "
Oct 30 19:15:30.834326 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.834252    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-workload-certs\") pod \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") "
Oct 30 19:15:30.834326 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.834299    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-podinfo\") pod \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") "
Oct 30 19:15:30.834499 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.834358    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istiod-ca-cert\") pod \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") "
Oct 30 19:15:30.834499 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.834387    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a0996ab-5aaa-4b71-b064-87d9d1572d13-kube-api-access-vsrxj\" (UniqueName: \"kubernetes.io/projected/4a0996ab-5aaa-4b71-b064-87d9d1572d13-kube-api-access-vsrxj\") pod \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") "
Oct 30 19:15:30.834499 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.834464    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-data\") pod \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") "
Oct 30 19:15:30.834627 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.834502    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-credential-socket\") pod \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") "
Oct 30 19:15:30.834690 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.834652    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-workload-socket\") pod \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\" (UID: \"4a0996ab-5aaa-4b71-b064-87d9d1572d13\") "
Oct 30 19:15:30.835210 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.834878    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-credential-socket" pod "4a0996ab-5aaa-4b71-b064-87d9d1572d13" (UID: "4a0996ab-5aaa-4b71-b064-87d9d1572d13"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:30.835210 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.835175    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-workload-socket" pod "4a0996ab-5aaa-4b71-b064-87d9d1572d13" (UID: "4a0996ab-5aaa-4b71-b064-87d9d1572d13"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:30.835210 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.835202    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-data" pod "4a0996ab-5aaa-4b71-b064-87d9d1572d13" (UID: "4a0996ab-5aaa-4b71-b064-87d9d1572d13"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:30.835635 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.835369    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:30.835635 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.835387    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:30.835635 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.835404    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:30.835635 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.835612    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istiod-ca-cert" pod "4a0996ab-5aaa-4b71-b064-87d9d1572d13" (UID: "4a0996ab-5aaa-4b71-b064-87d9d1572d13"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:30.835880 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.835675    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-workload-certs" pod "4a0996ab-5aaa-4b71-b064-87d9d1572d13" (UID: "4a0996ab-5aaa-4b71-b064-87d9d1572d13"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:30.836919 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.836880    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-envoy" pod "4a0996ab-5aaa-4b71-b064-87d9d1572d13" (UID: "4a0996ab-5aaa-4b71-b064-87d9d1572d13"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:30.837698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.837657    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a0996ab-5aaa-4b71-b064-87d9d1572d13-kube-api-access-vsrxj" pod "4a0996ab-5aaa-4b71-b064-87d9d1572d13" (UID: "4a0996ab-5aaa-4b71-b064-87d9d1572d13"). InnerVolumeSpecName "kube-api-access-vsrxj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:30.837992 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.837924    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-podinfo" pod "4a0996ab-5aaa-4b71-b064-87d9d1572d13" (UID: "4a0996ab-5aaa-4b71-b064-87d9d1572d13"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:15:30.838333 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.838294    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-token" pod "4a0996ab-5aaa-4b71-b064-87d9d1572d13" (UID: "4a0996ab-5aaa-4b71-b064-87d9d1572d13"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:30.840015 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.839985    5820 generic.go:356] "Generic (PLEG): container finished" podID="4a0996ab-5aaa-4b71-b064-87d9d1572d13" containerID="eeafa7b7322daa533729b4543a212ac1f6cc19b8ff9cf0832fd11353f8675506" exitCode=0
Oct 30 19:15:30.840015 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.839997    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6" event={"ID":"4a0996ab-5aaa-4b71-b064-87d9d1572d13","Type":"ContainerDied","Data":"eeafa7b7322daa533729b4543a212ac1f6cc19b8ff9cf0832fd11353f8675506"}
Oct 30 19:15:30.840175 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.840035    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"
Oct 30 19:15:30.840175 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.840072    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6" event={"ID":"4a0996ab-5aaa-4b71-b064-87d9d1572d13","Type":"ContainerDied","Data":"f08f3689ecb2162ad41fd9d2d8751faa0ac34544fab1a65f63b5aad04569285f"}
Oct 30 19:15:30.840175 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.840107    5820 scope.go:122] "RemoveContainer" containerID="eeafa7b7322daa533729b4543a212ac1f6cc19b8ff9cf0832fd11353f8675506"
Oct 30 19:15:30.867070 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.867046    5820 scope.go:122] "RemoveContainer" containerID="fa6c9f30b0bb40402b99756c717058cc797c78f47ad97cb081f4876bd879ea3c"
Oct 30 19:15:30.876909 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.876885    5820 scope.go:122] "RemoveContainer" containerID="eeafa7b7322daa533729b4543a212ac1f6cc19b8ff9cf0832fd11353f8675506"
Oct 30 19:15:30.877326 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:15:30.877285    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eeafa7b7322daa533729b4543a212ac1f6cc19b8ff9cf0832fd11353f8675506\": container with ID starting with eeafa7b7322daa533729b4543a212ac1f6cc19b8ff9cf0832fd11353f8675506 not found: ID does not exist" containerID="eeafa7b7322daa533729b4543a212ac1f6cc19b8ff9cf0832fd11353f8675506"
Oct 30 19:15:30.877433 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.877325    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeafa7b7322daa533729b4543a212ac1f6cc19b8ff9cf0832fd11353f8675506"} err="failed to get container status \"eeafa7b7322daa533729b4543a212ac1f6cc19b8ff9cf0832fd11353f8675506\": rpc error: code = NotFound desc = could not find container \"eeafa7b7322daa533729b4543a212ac1f6cc19b8ff9cf0832fd11353f8675506\": container with ID starting with eeafa7b7322daa533729b4543a212ac1f6cc19b8ff9cf0832fd11353f8675506 not found: ID does not exist"
Oct 30 19:15:30.877433 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.877352    5820 scope.go:122] "RemoveContainer" containerID="fa6c9f30b0bb40402b99756c717058cc797c78f47ad97cb081f4876bd879ea3c"
Oct 30 19:15:30.877734 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:15:30.877714    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa6c9f30b0bb40402b99756c717058cc797c78f47ad97cb081f4876bd879ea3c\": container with ID starting with fa6c9f30b0bb40402b99756c717058cc797c78f47ad97cb081f4876bd879ea3c not found: ID does not exist" containerID="fa6c9f30b0bb40402b99756c717058cc797c78f47ad97cb081f4876bd879ea3c"
Oct 30 19:15:30.877804 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.877744    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa6c9f30b0bb40402b99756c717058cc797c78f47ad97cb081f4876bd879ea3c"} err="failed to get container status \"fa6c9f30b0bb40402b99756c717058cc797c78f47ad97cb081f4876bd879ea3c\": rpc error: code = NotFound desc = could not find container \"fa6c9f30b0bb40402b99756c717058cc797c78f47ad97cb081f4876bd879ea3c\": container with ID starting with fa6c9f30b0bb40402b99756c717058cc797c78f47ad97cb081f4876bd879ea3c not found: ID does not exist"
Oct 30 19:15:30.936258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.936231    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:30.936258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.936255    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:30.936499 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.936268    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a0996ab-5aaa-4b71-b064-87d9d1572d13-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:30.936499 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.936285    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:30.936499 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.936298    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a0996ab-5aaa-4b71-b064-87d9d1572d13-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:30.936499 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:30.936312    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsrxj\" (UniqueName: \"kubernetes.io/projected/4a0996ab-5aaa-4b71-b064-87d9d1572d13-kube-api-access-vsrxj\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:31.026325 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:31.026244    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"]
Oct 30 19:15:31.032197 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:31.032166    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-0be4a4-openshift-default-7b76544cc7-m2rm6"]
Oct 30 19:15:31.420700 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:31.420673    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:31.421127 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:31.421071    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:31.421698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:31.421681    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:15:31.438028 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:31.438003    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4a0996ab-5aaa-4b71-b064-87d9d1572d13" path="/var/lib/kubelet/pods/4a0996ab-5aaa-4b71-b064-87d9d1572d13/volumes"
Oct 30 19:15:33.424983 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:33.422343    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:37.903896 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:37.903857    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l"]
Oct 30 19:15:38.015688 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:38.015651    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8"]
Oct 30 19:15:38.222768 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:15:38.222674    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:15:38.323246 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:38.323212    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:38.505425 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:38.505331    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:38.682002 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:38.681969    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:42.904178 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:42.904117    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l" podUID="6da26a61-a369-4f91-ad4d-d4b8510b6055" containerName="payload-processing" containerID="cri-o://f90f9fdce9cd7fe9898f09fa3331f02699c8833dc3988ab5fb2e7fee0a360da6" gracePeriod=25
Oct 30 19:15:43.016253 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:43.016086    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8" podUID="47db0d9f-9872-494d-aa71-f541802898f7" containerName="payload-pre-processing" containerID="cri-o://a94c20b1dbc693c12c2aa306c76ffcf8a9adc7796780033afa182b0f3169b0b3" gracePeriod=25
Oct 30 19:15:43.395975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:43.394732    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l"
Oct 30 19:15:43.429717 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:43.429679    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6da26a61-a369-4f91-ad4d-d4b8510b6055-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6da26a61-a369-4f91-ad4d-d4b8510b6055-plugins-config-volume\") pod \"6da26a61-a369-4f91-ad4d-d4b8510b6055\" (UID: \"6da26a61-a369-4f91-ad4d-d4b8510b6055\") "
Oct 30 19:15:43.429897 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:43.429729    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6da26a61-a369-4f91-ad4d-d4b8510b6055-kube-api-access-5xw8g\" (UniqueName: \"kubernetes.io/projected/6da26a61-a369-4f91-ad4d-d4b8510b6055-kube-api-access-5xw8g\") pod \"6da26a61-a369-4f91-ad4d-d4b8510b6055\" (UID: \"6da26a61-a369-4f91-ad4d-d4b8510b6055\") "
Oct 30 19:15:43.431877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:43.430532    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6da26a61-a369-4f91-ad4d-d4b8510b6055-plugins-config-volume" pod "6da26a61-a369-4f91-ad4d-d4b8510b6055" (UID: "6da26a61-a369-4f91-ad4d-d4b8510b6055"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:43.481985 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:43.480132    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6da26a61-a369-4f91-ad4d-d4b8510b6055-kube-api-access-5xw8g" pod "6da26a61-a369-4f91-ad4d-d4b8510b6055" (UID: "6da26a61-a369-4f91-ad4d-d4b8510b6055"). InnerVolumeSpecName "kube-api-access-5xw8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:43.500631 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:43.500609    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8"
Oct 30 19:15:43.531428 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:43.531404    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/47db0d9f-9872-494d-aa71-f541802898f7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47db0d9f-9872-494d-aa71-f541802898f7-plugins-config-volume\") pod \"47db0d9f-9872-494d-aa71-f541802898f7\" (UID: \"47db0d9f-9872-494d-aa71-f541802898f7\") "
Oct 30 19:15:43.531610 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:43.531600    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47db0d9f-9872-494d-aa71-f541802898f7-kube-api-access-66d8g\" (UniqueName: \"kubernetes.io/projected/47db0d9f-9872-494d-aa71-f541802898f7-kube-api-access-66d8g\") pod \"47db0d9f-9872-494d-aa71-f541802898f7\" (UID: \"47db0d9f-9872-494d-aa71-f541802898f7\") "
Oct 30 19:15:43.531995 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:43.531777    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47db0d9f-9872-494d-aa71-f541802898f7-plugins-config-volume" pod "47db0d9f-9872-494d-aa71-f541802898f7" (UID: "47db0d9f-9872-494d-aa71-f541802898f7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:43.532836 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:43.532522    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xw8g\" (UniqueName: \"kubernetes.io/projected/6da26a61-a369-4f91-ad4d-d4b8510b6055-kube-api-access-5xw8g\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:43.532836 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:43.532548    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47db0d9f-9872-494d-aa71-f541802898f7-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:43.532836 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:43.532562    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6da26a61-a369-4f91-ad4d-d4b8510b6055-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:43.570647 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:43.563733    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47db0d9f-9872-494d-aa71-f541802898f7-kube-api-access-66d8g" pod "47db0d9f-9872-494d-aa71-f541802898f7" (UID: "47db0d9f-9872-494d-aa71-f541802898f7"). InnerVolumeSpecName "kube-api-access-66d8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:43.570647 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:43.570251    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:43.655977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:43.642473    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66d8g\" (UniqueName: \"kubernetes.io/projected/47db0d9f-9872-494d-aa71-f541802898f7-kube-api-access-66d8g\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:44.001448 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:44.001415    5820 generic.go:356] "Generic (PLEG): container finished" podID="47db0d9f-9872-494d-aa71-f541802898f7" containerID="a94c20b1dbc693c12c2aa306c76ffcf8a9adc7796780033afa182b0f3169b0b3" exitCode=0
Oct 30 19:15:44.001928 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:44.001477    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8"
Oct 30 19:15:44.001928 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:44.001500    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8" event={"ID":"47db0d9f-9872-494d-aa71-f541802898f7","Type":"ContainerDied","Data":"a94c20b1dbc693c12c2aa306c76ffcf8a9adc7796780033afa182b0f3169b0b3"}
Oct 30 19:15:44.001928 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:44.001561    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8" event={"ID":"47db0d9f-9872-494d-aa71-f541802898f7","Type":"ContainerDied","Data":"ea199202bbbdeb6304ebeb67dd82bf475ae86d848134da9c60f84274331bf111"}
Oct 30 19:15:44.001928 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:44.001586    5820 scope.go:122] "RemoveContainer" containerID="a94c20b1dbc693c12c2aa306c76ffcf8a9adc7796780033afa182b0f3169b0b3"
Oct 30 19:15:44.003268 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:44.003247    5820 generic.go:356] "Generic (PLEG): container finished" podID="6da26a61-a369-4f91-ad4d-d4b8510b6055" containerID="f90f9fdce9cd7fe9898f09fa3331f02699c8833dc3988ab5fb2e7fee0a360da6" exitCode=0
Oct 30 19:15:44.003388 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:44.003294    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l"
Oct 30 19:15:44.003388 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:44.003312    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l" event={"ID":"6da26a61-a369-4f91-ad4d-d4b8510b6055","Type":"ContainerDied","Data":"f90f9fdce9cd7fe9898f09fa3331f02699c8833dc3988ab5fb2e7fee0a360da6"}
Oct 30 19:15:44.003388 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:44.003352    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l" event={"ID":"6da26a61-a369-4f91-ad4d-d4b8510b6055","Type":"ContainerDied","Data":"d080ca4851626045d99bcc1bd6a9ba552d583156bf1005887c42f63ba051c4d5"}
Oct 30 19:15:44.013447 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:44.011998    5820 scope.go:122] "RemoveContainer" containerID="a94c20b1dbc693c12c2aa306c76ffcf8a9adc7796780033afa182b0f3169b0b3"
Oct 30 19:15:44.013447 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:15:44.012399    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a94c20b1dbc693c12c2aa306c76ffcf8a9adc7796780033afa182b0f3169b0b3\": container with ID starting with a94c20b1dbc693c12c2aa306c76ffcf8a9adc7796780033afa182b0f3169b0b3 not found: ID does not exist" containerID="a94c20b1dbc693c12c2aa306c76ffcf8a9adc7796780033afa182b0f3169b0b3"
Oct 30 19:15:44.013447 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:44.012434    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a94c20b1dbc693c12c2aa306c76ffcf8a9adc7796780033afa182b0f3169b0b3"} err="failed to get container status \"a94c20b1dbc693c12c2aa306c76ffcf8a9adc7796780033afa182b0f3169b0b3\": rpc error: code = NotFound desc = could not find container \"a94c20b1dbc693c12c2aa306c76ffcf8a9adc7796780033afa182b0f3169b0b3\": container with ID starting with a94c20b1dbc693c12c2aa306c76ffcf8a9adc7796780033afa182b0f3169b0b3 not found: ID does not exist"
Oct 30 19:15:44.013447 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:44.012473    5820 scope.go:122] "RemoveContainer" containerID="f90f9fdce9cd7fe9898f09fa3331f02699c8833dc3988ab5fb2e7fee0a360da6"
Oct 30 19:15:44.021649 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:44.021617    5820 scope.go:122] "RemoveContainer" containerID="f90f9fdce9cd7fe9898f09fa3331f02699c8833dc3988ab5fb2e7fee0a360da6"
Oct 30 19:15:44.021961 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:15:44.021929    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f90f9fdce9cd7fe9898f09fa3331f02699c8833dc3988ab5fb2e7fee0a360da6\": container with ID starting with f90f9fdce9cd7fe9898f09fa3331f02699c8833dc3988ab5fb2e7fee0a360da6 not found: ID does not exist" containerID="f90f9fdce9cd7fe9898f09fa3331f02699c8833dc3988ab5fb2e7fee0a360da6"
Oct 30 19:15:44.022043 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:44.022030    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f90f9fdce9cd7fe9898f09fa3331f02699c8833dc3988ab5fb2e7fee0a360da6"} err="failed to get container status \"f90f9fdce9cd7fe9898f09fa3331f02699c8833dc3988ab5fb2e7fee0a360da6\": rpc error: code = NotFound desc = could not find container \"f90f9fdce9cd7fe9898f09fa3331f02699c8833dc3988ab5fb2e7fee0a360da6\": container with ID starting with f90f9fdce9cd7fe9898f09fa3331f02699c8833dc3988ab5fb2e7fee0a360da6 not found: ID does not exist"
Oct 30 19:15:44.167938 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:44.167848    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l"]
Oct 30 19:15:44.217436 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:44.217393    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-cf43e7-6cfc57d666-2t99l"]
Oct 30 19:15:44.281479 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:44.281441    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8"]
Oct 30 19:15:44.283812 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:44.283783    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-cf43e7-56ff648b59-wdqg8"]
Oct 30 19:15:45.142139 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:45.142088    5820 patch_prober.go:29] interesting pod/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:15:45.142696 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:45.142164    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b" podUID="126f2139-f087-4162-af9d-ca9edd5c83da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:15:45.441223 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:45.441130    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="47db0d9f-9872-494d-aa71-f541802898f7" path="/var/lib/kubelet/pods/47db0d9f-9872-494d-aa71-f541802898f7/volumes"
Oct 30 19:15:45.441658 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:45.441636    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6da26a61-a369-4f91-ad4d-d4b8510b6055" path="/var/lib/kubelet/pods/6da26a61-a369-4f91-ad4d-d4b8510b6055/volumes"
Oct 30 19:15:48.036906 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:48.036813    5820 generic.go:356] "Generic (PLEG): container finished" podID="126f2139-f087-4162-af9d-ca9edd5c83da" containerID="2531213a74bf757ed94e1886fbd2cf3e7bc04993c5c4a70823a4401f767f3e6c" exitCode=0
Oct 30 19:15:48.037510 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:48.036896    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b" event={"ID":"126f2139-f087-4162-af9d-ca9edd5c83da","Type":"ContainerDied","Data":"2531213a74bf757ed94e1886fbd2cf3e7bc04993c5c4a70823a4401f767f3e6c"}
Oct 30 19:15:48.037510 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:48.037398    5820 scope.go:122] "RemoveContainer" containerID="2531213a74bf757ed94e1886fbd2cf3e7bc04993c5c4a70823a4401f767f3e6c"
Oct 30 19:15:48.038988 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:48.038942    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:15:48.039070 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:48.039038    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:15:48.039109 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:48.039073    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:15:48.491403 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:48.491304    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:49.067299 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:49.067260    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b" event={"ID":"126f2139-f087-4162-af9d-ca9edd5c83da","Type":"ContainerStarted","Data":"99c82650cfe031bcb04f334773c9e31106b3c1b4eac2b49e387f27cb568c2ef9"}
Oct 30 19:15:49.068910 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:49.068887    5820 generic.go:356] "Generic (PLEG): container finished" podID="17ba2a01-0007-4f37-b1a9-464910fd82e6" containerID="b38c1203a7f746311664f4a5481d28f31e69bb32071a0d451a1670d375f31cd8" exitCode=0
Oct 30 19:15:49.069056 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:49.068988    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf" event={"ID":"17ba2a01-0007-4f37-b1a9-464910fd82e6","Type":"ContainerDied","Data":"b38c1203a7f746311664f4a5481d28f31e69bb32071a0d451a1670d375f31cd8"}
Oct 30 19:15:49.069310 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:49.069296    5820 scope.go:122] "RemoveContainer" containerID="b38c1203a7f746311664f4a5481d28f31e69bb32071a0d451a1670d375f31cd8"
Oct 30 19:15:49.070573 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:49.070542    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:15:49.070656 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:49.070609    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:15:49.070656 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:49.070642    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:15:49.945897 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:49.945859    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:15:49.963354 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:49.961071    5820 patch_prober.go:29] interesting pod/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 30 19:15:49.963720 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:49.963688    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b" podUID="126f2139-f087-4162-af9d-ca9edd5c83da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 30 19:15:50.092082 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:50.092038    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf" event={"ID":"17ba2a01-0007-4f37-b1a9-464910fd82e6","Type":"ContainerStarted","Data":"a557ecfed49b1f9da927736ecd1eb28440a36b8c36e15ed5cd3573e3e2b69b9a"}
Oct 30 19:15:50.763995 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:50.763235    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:15:50.768972 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:50.765385    5820 patch_prober.go:29] interesting pod/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused" start-of-body=
Oct 30 19:15:50.768972 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:50.765441    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf" podUID="17ba2a01-0007-4f37-b1a9-464910fd82e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused"
Oct 30 19:15:50.954975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:50.947323    5820 patch_prober.go:29] interesting pod/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 30 19:15:50.954975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:50.947399    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b" podUID="126f2139-f087-4162-af9d-ca9edd5c83da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 30 19:15:51.761998 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:51.761284    5820 patch_prober.go:29] interesting pod/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused" start-of-body=
Oct 30 19:15:51.761998 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:51.761343    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf" podUID="17ba2a01-0007-4f37-b1a9-464910fd82e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused"
Oct 30 19:15:51.961976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:51.961073    5820 patch_prober.go:29] interesting pod/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 30 19:15:51.961976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:51.961136    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b" podUID="126f2139-f087-4162-af9d-ca9edd5c83da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 30 19:15:52.763730 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:52.763295    5820 patch_prober.go:29] interesting pod/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused" start-of-body=
Oct 30 19:15:52.763730 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:52.763360    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf" podUID="17ba2a01-0007-4f37-b1a9-464910fd82e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused"
Oct 30 19:15:52.949677 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:52.949256    5820 patch_prober.go:29] interesting pod/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 30 19:15:52.949677 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:52.949312    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b" podUID="126f2139-f087-4162-af9d-ca9edd5c83da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 30 19:15:53.291920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:53.291881    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:53.767626 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:53.767590    5820 patch_prober.go:29] interesting pod/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused" start-of-body=
Oct 30 19:15:53.768315 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:53.768285    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf" podUID="17ba2a01-0007-4f37-b1a9-464910fd82e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused"
Oct 30 19:15:53.955118 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:53.955083    5820 patch_prober.go:29] interesting pod/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 30 19:15:53.955439 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:53.955394    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b" podUID="126f2139-f087-4162-af9d-ca9edd5c83da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 30 19:15:54.760349 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:54.760311    5820 patch_prober.go:29] interesting pod/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused" start-of-body=
Oct 30 19:15:54.760576 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:54.760420    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf" podUID="17ba2a01-0007-4f37-b1a9-464910fd82e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused"
Oct 30 19:15:54.963491 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:54.963460    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:15:54.963906 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:54.963859    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:15:54.964677 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:54.964659    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:15:55.179541 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:55.179511    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:15:55.764663 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:55.764631    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:15:56.131046 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:56.131017    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:15:57.477283 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:57.477243    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:58.034214 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:58.034180    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:58.210071 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:58.210034    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:59.116057 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:59.116021    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:59.373427 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:59.373346    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:59.577165 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:15:59.577129    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:00.095933 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:00.095902    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:00.449326 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:00.449245    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:04.902625 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:04.902583    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:05.211446 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:05.211351    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:08.254896 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:16:08.254845    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:16:11.496542 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:11.496503    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:16.544326 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:16.544238    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:19.382552 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:19.382510    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:23.452671 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:23.452630    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:28.767701 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:28.767667    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:30.494940 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:30.494902    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf"]
Oct 30 19:16:30.599878 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:30.599845    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6"]
Oct 30 19:16:30.763540 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:30.763448    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:30.947002 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:30.946882    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:31.124585 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:31.124549    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:31.310401 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:31.310353    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:33.161403 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:33.161369    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"]
Oct 30 19:16:33.161807 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:33.161582    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw" podUID="c3b24a33-1a96-41be-abca-2a75550b1c46" containerName="maas-api" containerID="cri-o://016faa8c3b57b74651c932e32835d1aa47bd4ad281698bec4f0fbb230e5f378c" gracePeriod=30
Oct 30 19:16:33.229314 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:33.229280    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d"]
Oct 30 19:16:33.346803 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:33.346768    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:33.394564 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:33.394289    5820 generic.go:356] "Generic (PLEG): container finished" podID="c3b24a33-1a96-41be-abca-2a75550b1c46" containerID="016faa8c3b57b74651c932e32835d1aa47bd4ad281698bec4f0fbb230e5f378c" exitCode=0
Oct 30 19:16:33.394564 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:33.394478    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw" event={"ID":"c3b24a33-1a96-41be-abca-2a75550b1c46","Type":"ContainerDied","Data":"016faa8c3b57b74651c932e32835d1aa47bd4ad281698bec4f0fbb230e5f378c"}
Oct 30 19:16:33.492720 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:33.492692    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"
Oct 30 19:16:33.566830 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:33.566797    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c3b24a33-1a96-41be-abca-2a75550b1c46-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3b24a33-1a96-41be-abca-2a75550b1c46-maas-api-tls\") pod \"c3b24a33-1a96-41be-abca-2a75550b1c46\" (UID: \"c3b24a33-1a96-41be-abca-2a75550b1c46\") "
Oct 30 19:16:33.567003 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:33.566865    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c3b24a33-1a96-41be-abca-2a75550b1c46-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3b24a33-1a96-41be-abca-2a75550b1c46-metrics-certs\") pod \"c3b24a33-1a96-41be-abca-2a75550b1c46\" (UID: \"c3b24a33-1a96-41be-abca-2a75550b1c46\") "
Oct 30 19:16:33.567003 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:33.566926    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c3b24a33-1a96-41be-abca-2a75550b1c46-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3b24a33-1a96-41be-abca-2a75550b1c46-openshift-service-ca\") pod \"c3b24a33-1a96-41be-abca-2a75550b1c46\" (UID: \"c3b24a33-1a96-41be-abca-2a75550b1c46\") "
Oct 30 19:16:33.567003 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:33.566998    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3b24a33-1a96-41be-abca-2a75550b1c46-kube-api-access-gbcrn\" (UniqueName: \"kubernetes.io/projected/c3b24a33-1a96-41be-abca-2a75550b1c46-kube-api-access-gbcrn\") pod \"c3b24a33-1a96-41be-abca-2a75550b1c46\" (UID: \"c3b24a33-1a96-41be-abca-2a75550b1c46\") "
Oct 30 19:16:33.567332 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:33.567304    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3b24a33-1a96-41be-abca-2a75550b1c46-openshift-service-ca" pod "c3b24a33-1a96-41be-abca-2a75550b1c46" (UID: "c3b24a33-1a96-41be-abca-2a75550b1c46"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:33.567517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:33.567502    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3b24a33-1a96-41be-abca-2a75550b1c46-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:33.569168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:33.569140    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3b24a33-1a96-41be-abca-2a75550b1c46-metrics-certs" pod "c3b24a33-1a96-41be-abca-2a75550b1c46" (UID: "c3b24a33-1a96-41be-abca-2a75550b1c46"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:33.569168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:33.569154    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3b24a33-1a96-41be-abca-2a75550b1c46-maas-api-tls" pod "c3b24a33-1a96-41be-abca-2a75550b1c46" (UID: "c3b24a33-1a96-41be-abca-2a75550b1c46"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:33.569292 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:33.569197    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3b24a33-1a96-41be-abca-2a75550b1c46-kube-api-access-gbcrn" pod "c3b24a33-1a96-41be-abca-2a75550b1c46" (UID: "c3b24a33-1a96-41be-abca-2a75550b1c46"). InnerVolumeSpecName "kube-api-access-gbcrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:33.575001 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:33.574975    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:33.668805 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:33.668774    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbcrn\" (UniqueName: \"kubernetes.io/projected/c3b24a33-1a96-41be-abca-2a75550b1c46-kube-api-access-gbcrn\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:33.668805 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:33.668804    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3b24a33-1a96-41be-abca-2a75550b1c46-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:33.669038 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:33.668815    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3b24a33-1a96-41be-abca-2a75550b1c46-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:33.766458 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:33.766371    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:34.401980 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:34.401932    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw" event={"ID":"c3b24a33-1a96-41be-abca-2a75550b1c46","Type":"ContainerDied","Data":"830427731e81fcd8f56dc796ce422d216c660c4c0a241f3c12621363ed262b82"}
Oct 30 19:16:34.401980 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:34.401983    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"
Oct 30 19:16:34.402453 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:34.402029    5820 scope.go:122] "RemoveContainer" containerID="016faa8c3b57b74651c932e32835d1aa47bd4ad281698bec4f0fbb230e5f378c"
Oct 30 19:16:34.569862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:34.569824    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"]
Oct 30 19:16:34.577329 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:34.577303    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-cf43e7-cdf6d59b9-2trsw"]
Oct 30 19:16:35.023287 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:35.023256    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:35.202367 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:35.202331    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:35.395696 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:35.395660    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:35.436863 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:35.436834    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c3b24a33-1a96-41be-abca-2a75550b1c46" path="/var/lib/kubelet/pods/c3b24a33-1a96-41be-abca-2a75550b1c46/volumes"
Oct 30 19:16:35.495742 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:35.495708    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf" podUID="f04590de-490c-4c27-a42a-b17378b00e84" containerName="payload-processing" containerID="cri-o://e8a06fef9cc531ded714873f9c16b02199c9abc0f822dbca596cc9bfa97c556e" gracePeriod=25
Oct 30 19:16:35.600628 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:35.600584    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6" podUID="bdbab4fc-60b7-4179-a386-7a8f96814802" containerName="payload-pre-processing" containerID="cri-o://e183d3abbee812c489abacd0c5f6dff711772f80fccfaab5c25fe03b31ecb1ff" gracePeriod=25
Oct 30 19:16:35.761618 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:35.761585    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf"
Oct 30 19:16:35.855932 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:35.855909    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6"
Oct 30 19:16:35.902551 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:35.902518    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f04590de-490c-4c27-a42a-b17378b00e84-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f04590de-490c-4c27-a42a-b17378b00e84-plugins-config-volume\") pod \"f04590de-490c-4c27-a42a-b17378b00e84\" (UID: \"f04590de-490c-4c27-a42a-b17378b00e84\") "
Oct 30 19:16:35.902718 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:35.902592    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f04590de-490c-4c27-a42a-b17378b00e84-kube-api-access-lqllb\" (UniqueName: \"kubernetes.io/projected/f04590de-490c-4c27-a42a-b17378b00e84-kube-api-access-lqllb\") pod \"f04590de-490c-4c27-a42a-b17378b00e84\" (UID: \"f04590de-490c-4c27-a42a-b17378b00e84\") "
Oct 30 19:16:35.902999 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:35.902937    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f04590de-490c-4c27-a42a-b17378b00e84-plugins-config-volume" pod "f04590de-490c-4c27-a42a-b17378b00e84" (UID: "f04590de-490c-4c27-a42a-b17378b00e84"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:35.903266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:35.903247    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f04590de-490c-4c27-a42a-b17378b00e84-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:35.905197 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:35.905163    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f04590de-490c-4c27-a42a-b17378b00e84-kube-api-access-lqllb" pod "f04590de-490c-4c27-a42a-b17378b00e84" (UID: "f04590de-490c-4c27-a42a-b17378b00e84"). InnerVolumeSpecName "kube-api-access-lqllb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:36.007718 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.007619    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bdbab4fc-60b7-4179-a386-7a8f96814802-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdbab4fc-60b7-4179-a386-7a8f96814802-plugins-config-volume\") pod \"bdbab4fc-60b7-4179-a386-7a8f96814802\" (UID: \"bdbab4fc-60b7-4179-a386-7a8f96814802\") "
Oct 30 19:16:36.008072 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.008047    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bdbab4fc-60b7-4179-a386-7a8f96814802-kube-api-access-8j59d\" (UniqueName: \"kubernetes.io/projected/bdbab4fc-60b7-4179-a386-7a8f96814802-kube-api-access-8j59d\") pod \"bdbab4fc-60b7-4179-a386-7a8f96814802\" (UID: \"bdbab4fc-60b7-4179-a386-7a8f96814802\") "
Oct 30 19:16:36.008708 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.008686    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqllb\" (UniqueName: \"kubernetes.io/projected/f04590de-490c-4c27-a42a-b17378b00e84-kube-api-access-lqllb\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:36.009490 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.009461    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdbab4fc-60b7-4179-a386-7a8f96814802-plugins-config-volume" pod "bdbab4fc-60b7-4179-a386-7a8f96814802" (UID: "bdbab4fc-60b7-4179-a386-7a8f96814802"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:36.011778 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.011742    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdbab4fc-60b7-4179-a386-7a8f96814802-kube-api-access-8j59d" pod "bdbab4fc-60b7-4179-a386-7a8f96814802" (UID: "bdbab4fc-60b7-4179-a386-7a8f96814802"). InnerVolumeSpecName "kube-api-access-8j59d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:36.087563 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.087516    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:36.110380 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.110351    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8j59d\" (UniqueName: \"kubernetes.io/projected/bdbab4fc-60b7-4179-a386-7a8f96814802-kube-api-access-8j59d\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:36.110380 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.110379    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdbab4fc-60b7-4179-a386-7a8f96814802-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:36.416588 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.416553    5820 generic.go:356] "Generic (PLEG): container finished" podID="f04590de-490c-4c27-a42a-b17378b00e84" containerID="e8a06fef9cc531ded714873f9c16b02199c9abc0f822dbca596cc9bfa97c556e" exitCode=0
Oct 30 19:16:36.416818 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.416592    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf" event={"ID":"f04590de-490c-4c27-a42a-b17378b00e84","Type":"ContainerDied","Data":"e8a06fef9cc531ded714873f9c16b02199c9abc0f822dbca596cc9bfa97c556e"}
Oct 30 19:16:36.416818 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.416665    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf" event={"ID":"f04590de-490c-4c27-a42a-b17378b00e84","Type":"ContainerDied","Data":"249f554b2d91b1c000fc0c73a451e3a624ca01f5b3d735d3d1dda27ac9c3660b"}
Oct 30 19:16:36.416818 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.416608    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf"
Oct 30 19:16:36.416818 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.416702    5820 scope.go:122] "RemoveContainer" containerID="e8a06fef9cc531ded714873f9c16b02199c9abc0f822dbca596cc9bfa97c556e"
Oct 30 19:16:36.418914 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.418886    5820 generic.go:356] "Generic (PLEG): container finished" podID="bdbab4fc-60b7-4179-a386-7a8f96814802" containerID="e183d3abbee812c489abacd0c5f6dff711772f80fccfaab5c25fe03b31ecb1ff" exitCode=0
Oct 30 19:16:36.419082 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.418943    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6"
Oct 30 19:16:36.419156 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.419082    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6" event={"ID":"bdbab4fc-60b7-4179-a386-7a8f96814802","Type":"ContainerDied","Data":"e183d3abbee812c489abacd0c5f6dff711772f80fccfaab5c25fe03b31ecb1ff"}
Oct 30 19:16:36.419224 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.419163    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6" event={"ID":"bdbab4fc-60b7-4179-a386-7a8f96814802","Type":"ContainerDied","Data":"4414454dda51f743e44049906cab6de2683649c63709992e052e183a2971b200"}
Oct 30 19:16:36.428732 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.428636    5820 scope.go:122] "RemoveContainer" containerID="e8a06fef9cc531ded714873f9c16b02199c9abc0f822dbca596cc9bfa97c556e"
Oct 30 19:16:36.429034 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:16:36.429011    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8a06fef9cc531ded714873f9c16b02199c9abc0f822dbca596cc9bfa97c556e\": container with ID starting with e8a06fef9cc531ded714873f9c16b02199c9abc0f822dbca596cc9bfa97c556e not found: ID does not exist" containerID="e8a06fef9cc531ded714873f9c16b02199c9abc0f822dbca596cc9bfa97c556e"
Oct 30 19:16:36.429103 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.429046    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8a06fef9cc531ded714873f9c16b02199c9abc0f822dbca596cc9bfa97c556e"} err="failed to get container status \"e8a06fef9cc531ded714873f9c16b02199c9abc0f822dbca596cc9bfa97c556e\": rpc error: code = NotFound desc = could not find container \"e8a06fef9cc531ded714873f9c16b02199c9abc0f822dbca596cc9bfa97c556e\": container with ID starting with e8a06fef9cc531ded714873f9c16b02199c9abc0f822dbca596cc9bfa97c556e not found: ID does not exist"
Oct 30 19:16:36.429103 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.429089    5820 scope.go:122] "RemoveContainer" containerID="e183d3abbee812c489abacd0c5f6dff711772f80fccfaab5c25fe03b31ecb1ff"
Oct 30 19:16:36.440022 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.440006    5820 scope.go:122] "RemoveContainer" containerID="e183d3abbee812c489abacd0c5f6dff711772f80fccfaab5c25fe03b31ecb1ff"
Oct 30 19:16:36.440342 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:16:36.440311    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e183d3abbee812c489abacd0c5f6dff711772f80fccfaab5c25fe03b31ecb1ff\": container with ID starting with e183d3abbee812c489abacd0c5f6dff711772f80fccfaab5c25fe03b31ecb1ff not found: ID does not exist" containerID="e183d3abbee812c489abacd0c5f6dff711772f80fccfaab5c25fe03b31ecb1ff"
Oct 30 19:16:36.440403 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.440337    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e183d3abbee812c489abacd0c5f6dff711772f80fccfaab5c25fe03b31ecb1ff"} err="failed to get container status \"e183d3abbee812c489abacd0c5f6dff711772f80fccfaab5c25fe03b31ecb1ff\": rpc error: code = NotFound desc = could not find container \"e183d3abbee812c489abacd0c5f6dff711772f80fccfaab5c25fe03b31ecb1ff\": container with ID starting with e183d3abbee812c489abacd0c5f6dff711772f80fccfaab5c25fe03b31ecb1ff not found: ID does not exist"
Oct 30 19:16:36.597018 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.596982    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6"]
Oct 30 19:16:36.612939 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.612902    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-91cb32-5b5745fc79-wk7k6"]
Oct 30 19:16:36.756765 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.756682    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf"]
Oct 30 19:16:36.761249 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:36.761219    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-91cb32-88c9549c8-qvxqf"]
Oct 30 19:16:37.435994 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:37.435930    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bdbab4fc-60b7-4179-a386-7a8f96814802" path="/var/lib/kubelet/pods/bdbab4fc-60b7-4179-a386-7a8f96814802/volumes"
Oct 30 19:16:37.436429 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:37.436407    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f04590de-490c-4c27-a42a-b17378b00e84" path="/var/lib/kubelet/pods/f04590de-490c-4c27-a42a-b17378b00e84/volumes"
Oct 30 19:16:38.204086 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:16:38.204052    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:16:38.230069 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:38.229990    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d" podUID="337f5ca4-5214-48b2-a02b-671382ec43b2" containerName="payload-pre-processing" containerID="cri-o://6138b54e21a1f4a2fb4e88b6788994e706a8ba1785351d2f1e5964f8aae86820" gracePeriod=25
Oct 30 19:16:38.438986 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:38.436642    5820 generic.go:356] "Generic (PLEG): container finished" podID="337f5ca4-5214-48b2-a02b-671382ec43b2" containerID="6138b54e21a1f4a2fb4e88b6788994e706a8ba1785351d2f1e5964f8aae86820" exitCode=0
Oct 30 19:16:38.438986 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:38.436746    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d" event={"ID":"337f5ca4-5214-48b2-a02b-671382ec43b2","Type":"ContainerDied","Data":"6138b54e21a1f4a2fb4e88b6788994e706a8ba1785351d2f1e5964f8aae86820"}
Oct 30 19:16:38.496167 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:38.496135    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d"
Oct 30 19:16:38.637116 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:38.637082    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/337f5ca4-5214-48b2-a02b-671382ec43b2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/337f5ca4-5214-48b2-a02b-671382ec43b2-plugins-config-volume\") pod \"337f5ca4-5214-48b2-a02b-671382ec43b2\" (UID: \"337f5ca4-5214-48b2-a02b-671382ec43b2\") "
Oct 30 19:16:38.637311 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:38.637291    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/337f5ca4-5214-48b2-a02b-671382ec43b2-kube-api-access-x55j8\" (UniqueName: \"kubernetes.io/projected/337f5ca4-5214-48b2-a02b-671382ec43b2-kube-api-access-x55j8\") pod \"337f5ca4-5214-48b2-a02b-671382ec43b2\" (UID: \"337f5ca4-5214-48b2-a02b-671382ec43b2\") "
Oct 30 19:16:38.637437 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:38.637412    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/337f5ca4-5214-48b2-a02b-671382ec43b2-plugins-config-volume" pod "337f5ca4-5214-48b2-a02b-671382ec43b2" (UID: "337f5ca4-5214-48b2-a02b-671382ec43b2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:38.637687 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:38.637669    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/337f5ca4-5214-48b2-a02b-671382ec43b2-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:38.639612 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:38.639588    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/337f5ca4-5214-48b2-a02b-671382ec43b2-kube-api-access-x55j8" pod "337f5ca4-5214-48b2-a02b-671382ec43b2" (UID: "337f5ca4-5214-48b2-a02b-671382ec43b2"). InnerVolumeSpecName "kube-api-access-x55j8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:38.738647 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:38.738583    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x55j8\" (UniqueName: \"kubernetes.io/projected/337f5ca4-5214-48b2-a02b-671382ec43b2-kube-api-access-x55j8\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:39.444745 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:39.444469    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d" event={"ID":"337f5ca4-5214-48b2-a02b-671382ec43b2","Type":"ContainerDied","Data":"bdb9bdb57d1eb429cadab5f9e0521c9442242804a08f3558f0efaf95bb66dae7"}
Oct 30 19:16:39.444745 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:39.444500    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d"
Oct 30 19:16:39.444745 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:39.444543    5820 scope.go:122] "RemoveContainer" containerID="6138b54e21a1f4a2fb4e88b6788994e706a8ba1785351d2f1e5964f8aae86820"
Oct 30 19:16:39.611998 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:39.611930    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d"]
Oct 30 19:16:39.618972 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:39.618932    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-f554bb-59f76f77bc-gnn8d"]
Oct 30 19:16:40.602738 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:40.602702    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:41.437425 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:41.437392    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="337f5ca4-5214-48b2-a02b-671382ec43b2" path="/var/lib/kubelet/pods/337f5ca4-5214-48b2-a02b-671382ec43b2/volumes"
Oct 30 19:16:42.390471 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:42.390250    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:43.201817 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:43.201779    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:43.379373 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:43.379339    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:43.507145 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:43.507055    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"]
Oct 30 19:16:43.507585 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:43.507293    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq" podUID="fd708157-c511-4334-84c0-6d0a655e3502" containerName="istio-proxy" containerID="cri-o://66e767bb4219d89e63c9b136536add3a1cd8521432e97983928a8f60aace9844" gracePeriod=30
Oct 30 19:16:43.584031 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:43.582189    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:44.996302 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:44.996266    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:45.435478 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:45.435444    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:48.777849 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.777791    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:16:48.868565 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.868534    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-istio-data\") pod \"fd708157-c511-4334-84c0-6d0a655e3502\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") "
Oct 30 19:16:48.868730 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.868636    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd708157-c511-4334-84c0-6d0a655e3502-istio-token\" (UniqueName: \"kubernetes.io/projected/fd708157-c511-4334-84c0-6d0a655e3502-istio-token\") pod \"fd708157-c511-4334-84c0-6d0a655e3502\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") "
Oct 30 19:16:48.868730 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.868680    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fd708157-c511-4334-84c0-6d0a655e3502-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd708157-c511-4334-84c0-6d0a655e3502-istio-podinfo\") pod \"fd708157-c511-4334-84c0-6d0a655e3502\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") "
Oct 30 19:16:48.868813 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.868734    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-istio-envoy\") pod \"fd708157-c511-4334-84c0-6d0a655e3502\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") "
Oct 30 19:16:48.868813 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.868773    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-credential-socket\") pod \"fd708157-c511-4334-84c0-6d0a655e3502\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") "
Oct 30 19:16:48.868813 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.868780    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-istio-data" pod "fd708157-c511-4334-84c0-6d0a655e3502" (UID: "fd708157-c511-4334-84c0-6d0a655e3502"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:48.869047 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.868810    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-workload-socket\") pod \"fd708157-c511-4334-84c0-6d0a655e3502\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") "
Oct 30 19:16:48.869047 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.868846    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd708157-c511-4334-84c0-6d0a655e3502-kube-api-access-z6dpw\" (UniqueName: \"kubernetes.io/projected/fd708157-c511-4334-84c0-6d0a655e3502-kube-api-access-z6dpw\") pod \"fd708157-c511-4334-84c0-6d0a655e3502\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") "
Oct 30 19:16:48.869047 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.868883    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-workload-certs\") pod \"fd708157-c511-4334-84c0-6d0a655e3502\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") "
Oct 30 19:16:48.869047 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.868920    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd708157-c511-4334-84c0-6d0a655e3502-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd708157-c511-4334-84c0-6d0a655e3502-istiod-ca-cert\") pod \"fd708157-c511-4334-84c0-6d0a655e3502\" (UID: \"fd708157-c511-4334-84c0-6d0a655e3502\") "
Oct 30 19:16:48.869254 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.869048    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-credential-socket" pod "fd708157-c511-4334-84c0-6d0a655e3502" (UID: "fd708157-c511-4334-84c0-6d0a655e3502"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:48.869307 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.869247    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-workload-certs" pod "fd708157-c511-4334-84c0-6d0a655e3502" (UID: "fd708157-c511-4334-84c0-6d0a655e3502"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:48.869361 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.869329    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd708157-c511-4334-84c0-6d0a655e3502-istiod-ca-cert" pod "fd708157-c511-4334-84c0-6d0a655e3502" (UID: "fd708157-c511-4334-84c0-6d0a655e3502"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:48.869456 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.869438    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-workload-socket" pod "fd708157-c511-4334-84c0-6d0a655e3502" (UID: "fd708157-c511-4334-84c0-6d0a655e3502"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:48.869513 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.869493    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:48.869570 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.869514    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:48.869570 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.869532    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:48.869570 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.869551    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd708157-c511-4334-84c0-6d0a655e3502-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:48.871380 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.871351    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-istio-envoy" pod "fd708157-c511-4334-84c0-6d0a655e3502" (UID: "fd708157-c511-4334-84c0-6d0a655e3502"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:48.871520 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.871402    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd708157-c511-4334-84c0-6d0a655e3502-kube-api-access-z6dpw" pod "fd708157-c511-4334-84c0-6d0a655e3502" (UID: "fd708157-c511-4334-84c0-6d0a655e3502"). InnerVolumeSpecName "kube-api-access-z6dpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:48.871633 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.871613    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fd708157-c511-4334-84c0-6d0a655e3502-istio-podinfo" pod "fd708157-c511-4334-84c0-6d0a655e3502" (UID: "fd708157-c511-4334-84c0-6d0a655e3502"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:16:48.871693 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.871614    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd708157-c511-4334-84c0-6d0a655e3502-istio-token" pod "fd708157-c511-4334-84c0-6d0a655e3502" (UID: "fd708157-c511-4334-84c0-6d0a655e3502"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:48.971012 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.970980    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6dpw\" (UniqueName: \"kubernetes.io/projected/fd708157-c511-4334-84c0-6d0a655e3502-kube-api-access-z6dpw\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:48.971012 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.971008    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd708157-c511-4334-84c0-6d0a655e3502-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:48.971012 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.971018    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd708157-c511-4334-84c0-6d0a655e3502-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:48.971233 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.971027    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:48.971233 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:48.971038    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd708157-c511-4334-84c0-6d0a655e3502-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:49.525059 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:49.525009    5820 generic.go:356] "Generic (PLEG): container finished" podID="fd708157-c511-4334-84c0-6d0a655e3502" containerID="66e767bb4219d89e63c9b136536add3a1cd8521432e97983928a8f60aace9844" exitCode=0
Oct 30 19:16:49.525059 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:49.525050    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq" event={"ID":"fd708157-c511-4334-84c0-6d0a655e3502","Type":"ContainerDied","Data":"66e767bb4219d89e63c9b136536add3a1cd8521432e97983928a8f60aace9844"}
Oct 30 19:16:49.525281 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:49.525061    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"
Oct 30 19:16:49.525281 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:49.525099    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq" event={"ID":"fd708157-c511-4334-84c0-6d0a655e3502","Type":"ContainerDied","Data":"1d7862b3b7c9a567c8cec72df6b7f32aed823d3e5d0ec99a162dd9e85811ee6d"}
Oct 30 19:16:49.525281 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:49.525120    5820 scope.go:122] "RemoveContainer" containerID="66e767bb4219d89e63c9b136536add3a1cd8521432e97983928a8f60aace9844"
Oct 30 19:16:49.535030 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:49.535011    5820 scope.go:122] "RemoveContainer" containerID="6c51e3a8f0375ca46d81242e388b2c87378f348315273f154900a53e407d64cc"
Oct 30 19:16:49.543626 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:49.543607    5820 scope.go:122] "RemoveContainer" containerID="66e767bb4219d89e63c9b136536add3a1cd8521432e97983928a8f60aace9844"
Oct 30 19:16:49.543907 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:16:49.543884    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66e767bb4219d89e63c9b136536add3a1cd8521432e97983928a8f60aace9844\": container with ID starting with 66e767bb4219d89e63c9b136536add3a1cd8521432e97983928a8f60aace9844 not found: ID does not exist" containerID="66e767bb4219d89e63c9b136536add3a1cd8521432e97983928a8f60aace9844"
Oct 30 19:16:49.544024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:49.543918    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66e767bb4219d89e63c9b136536add3a1cd8521432e97983928a8f60aace9844"} err="failed to get container status \"66e767bb4219d89e63c9b136536add3a1cd8521432e97983928a8f60aace9844\": rpc error: code = NotFound desc = could not find container \"66e767bb4219d89e63c9b136536add3a1cd8521432e97983928a8f60aace9844\": container with ID starting with 66e767bb4219d89e63c9b136536add3a1cd8521432e97983928a8f60aace9844 not found: ID does not exist"
Oct 30 19:16:49.544024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:49.543943    5820 scope.go:122] "RemoveContainer" containerID="6c51e3a8f0375ca46d81242e388b2c87378f348315273f154900a53e407d64cc"
Oct 30 19:16:49.544250 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:16:49.544230    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c51e3a8f0375ca46d81242e388b2c87378f348315273f154900a53e407d64cc\": container with ID starting with 6c51e3a8f0375ca46d81242e388b2c87378f348315273f154900a53e407d64cc not found: ID does not exist" containerID="6c51e3a8f0375ca46d81242e388b2c87378f348315273f154900a53e407d64cc"
Oct 30 19:16:49.544296 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:49.544255    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c51e3a8f0375ca46d81242e388b2c87378f348315273f154900a53e407d64cc"} err="failed to get container status \"6c51e3a8f0375ca46d81242e388b2c87378f348315273f154900a53e407d64cc\": rpc error: code = NotFound desc = could not find container \"6c51e3a8f0375ca46d81242e388b2c87378f348315273f154900a53e407d64cc\": container with ID starting with 6c51e3a8f0375ca46d81242e388b2c87378f348315273f154900a53e407d64cc not found: ID does not exist"
Oct 30 19:16:49.593206 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:49.593167    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:49.753194 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:49.753086    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"]
Oct 30 19:16:49.787087 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:49.787040    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:49.802338 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:49.802313    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-cf43e7-openshift-default-6985994449-xk9kq"]
Oct 30 19:16:51.046686 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:51.046649    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:51.230704 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:51.230672    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:51.437620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:51.437588    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd708157-c511-4334-84c0-6d0a655e3502" path="/var/lib/kubelet/pods/fd708157-c511-4334-84c0-6d0a655e3502/volumes"
Oct 30 19:16:53.844309 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.844279    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"]
Oct 30 19:16:53.844818 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.844804    5820 cpu_manager.go:414] "containerMap: removing container" podUID="337f5ca4-5214-48b2-a02b-671382ec43b2" containerName="payload-pre-processing"
Oct 30 19:16:53.844864 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.844819    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="337f5ca4-5214-48b2-a02b-671382ec43b2" containerName="payload-pre-processing"
Oct 30 19:16:53.844864 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.844838    5820 cpu_manager.go:414] "containerMap: removing container" podUID="bdbab4fc-60b7-4179-a386-7a8f96814802" containerName="payload-pre-processing"
Oct 30 19:16:53.844864 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.844844    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bdbab4fc-60b7-4179-a386-7a8f96814802" containerName="payload-pre-processing"
Oct 30 19:16:53.845024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.844868    5820 cpu_manager.go:414] "containerMap: removing container" podUID="4a0996ab-5aaa-4b71-b064-87d9d1572d13" containerName="istio-proxy"
Oct 30 19:16:53.845024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.844874    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a0996ab-5aaa-4b71-b064-87d9d1572d13" containerName="istio-proxy"
Oct 30 19:16:53.845024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.844898    5820 cpu_manager.go:414] "containerMap: removing container" podUID="fd708157-c511-4334-84c0-6d0a655e3502" containerName="istio-proxy"
Oct 30 19:16:53.845024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.844903    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd708157-c511-4334-84c0-6d0a655e3502" containerName="istio-proxy"
Oct 30 19:16:53.845024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.844920    5820 cpu_manager.go:414] "containerMap: removing container" podUID="f04590de-490c-4c27-a42a-b17378b00e84" containerName="payload-processing"
Oct 30 19:16:53.845024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.844925    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f04590de-490c-4c27-a42a-b17378b00e84" containerName="payload-processing"
Oct 30 19:16:53.845024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.844931    5820 cpu_manager.go:414] "containerMap: removing container" podUID="fd708157-c511-4334-84c0-6d0a655e3502" containerName="istio-proxy"
Oct 30 19:16:53.845024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.844936    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd708157-c511-4334-84c0-6d0a655e3502" containerName="istio-proxy"
Oct 30 19:16:53.845024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.844943    5820 cpu_manager.go:414] "containerMap: removing container" podUID="c3b24a33-1a96-41be-abca-2a75550b1c46" containerName="maas-api"
Oct 30 19:16:53.845024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.844963    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3b24a33-1a96-41be-abca-2a75550b1c46" containerName="maas-api"
Oct 30 19:16:53.845024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.844975    5820 cpu_manager.go:414] "containerMap: removing container" podUID="6da26a61-a369-4f91-ad4d-d4b8510b6055" containerName="payload-processing"
Oct 30 19:16:53.845024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.844979    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6da26a61-a369-4f91-ad4d-d4b8510b6055" containerName="payload-processing"
Oct 30 19:16:53.845024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.844988    5820 cpu_manager.go:414] "containerMap: removing container" podUID="47db0d9f-9872-494d-aa71-f541802898f7" containerName="payload-pre-processing"
Oct 30 19:16:53.845024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.844993    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47db0d9f-9872-494d-aa71-f541802898f7" containerName="payload-pre-processing"
Oct 30 19:16:53.845517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.845063    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="47db0d9f-9872-494d-aa71-f541802898f7" containerName="payload-pre-processing"
Oct 30 19:16:53.845517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.845073    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="337f5ca4-5214-48b2-a02b-671382ec43b2" containerName="payload-pre-processing"
Oct 30 19:16:53.845517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.845079    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="bdbab4fc-60b7-4179-a386-7a8f96814802" containerName="payload-pre-processing"
Oct 30 19:16:53.845517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.845084    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a0996ab-5aaa-4b71-b064-87d9d1572d13" containerName="istio-proxy"
Oct 30 19:16:53.845517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.845093    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd708157-c511-4334-84c0-6d0a655e3502" containerName="istio-proxy"
Oct 30 19:16:53.845517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.845100    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="f04590de-490c-4c27-a42a-b17378b00e84" containerName="payload-processing"
Oct 30 19:16:53.845517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.845105    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3b24a33-1a96-41be-abca-2a75550b1c46" containerName="maas-api"
Oct 30 19:16:53.845517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.845113    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="6da26a61-a369-4f91-ad4d-d4b8510b6055" containerName="payload-processing"
Oct 30 19:16:53.850037 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.850014    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:53.948541 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.948498    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddw85\" (UniqueName: \"kubernetes.io/projected/c86c20cc-52fc-4ffb-aa87-46fcc272c457-kube-api-access-ddw85\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:53.948690 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.948567    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:53.948690 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.948637    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:53.948690 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.948671    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:53.948837 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.948821    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:53.948879 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.948861    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:53.949009 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.948991    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:53.949065 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.949030    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:53.949136 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.949119    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:53.957078 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:53.957050    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"]
Oct 30 19:16:54.050382 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.050345    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:54.050553 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.050404    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:54.050553 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.050447    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:54.050553 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.050539    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:54.050746 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.050576    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:54.050746 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.050679    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:54.050746 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.050717    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:54.050927 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.050779    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:54.050927 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.050826    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddw85\" (UniqueName: \"kubernetes.io/projected/c86c20cc-52fc-4ffb-aa87-46fcc272c457-kube-api-access-ddw85\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:54.050927 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.050909    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:54.051143 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.050987    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:54.051348 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.051219    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:54.051348 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.051363    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:54.051643 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.051521    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:54.053550 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.053524    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:54.053675 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.053599    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:54.111859 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.111799    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:54.112010 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.111906    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddw85\" (UniqueName: \"kubernetes.io/projected/c86c20cc-52fc-4ffb-aa87-46fcc272c457-kube-api-access-ddw85\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:54.164182 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.164152    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:54.602346 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.602317    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"]
Oct 30 19:16:54.604711 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:16:54.604678    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc86c20cc_52fc_4ffb_aa87_46fcc272c457.slice/crio-533ab8932b7b13081936d4f3c7d6f50890181759df79775a87ec0a661bcb516a WatchSource:0}: Error finding container 533ab8932b7b13081936d4f3c7d6f50890181759df79775a87ec0a661bcb516a: Status 404 returned error can't find the container with id 533ab8932b7b13081936d4f3c7d6f50890181759df79775a87ec0a661bcb516a
Oct 30 19:16:54.606968 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.606914    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:16:54.607092 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.607031    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:16:54.607092 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.607075    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:16:54.849362 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.849326    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"]
Oct 30 19:16:54.849886 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.849566    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x" podUID="174e6ae1-f7a9-4ee8-94b9-001dfbb9f387" containerName="maas-api" containerID="cri-o://83cb33008e2c921ddbe236313c964af510c77b500ae8c25781421209bda0d5d9" gracePeriod=30
Oct 30 19:16:54.997966 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:54.997923    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:55.163133 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.163107    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"
Oct 30 19:16:55.268985 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.266527    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-maas-api-tls\") pod \"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387\" (UID: \"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387\") "
Oct 30 19:16:55.268985 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.266592    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-metrics-certs\" (UniqueName: \"kubernetes.io/secret/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-metrics-certs\") pod \"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387\" (UID: \"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387\") "
Oct 30 19:16:55.268985 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.266709    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-openshift-service-ca\") pod \"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387\" (UID: \"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387\") "
Oct 30 19:16:55.268985 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.266906    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-kube-api-access-n64r2\" (UniqueName: \"kubernetes.io/projected/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-kube-api-access-n64r2\") pod \"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387\" (UID: \"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387\") "
Oct 30 19:16:55.270206 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.270174    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-openshift-service-ca" pod "174e6ae1-f7a9-4ee8-94b9-001dfbb9f387" (UID: "174e6ae1-f7a9-4ee8-94b9-001dfbb9f387"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:55.274015 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.273379    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:55.279286 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.279237    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-maas-api-tls" pod "174e6ae1-f7a9-4ee8-94b9-001dfbb9f387" (UID: "174e6ae1-f7a9-4ee8-94b9-001dfbb9f387"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:55.280400 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.280348    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-kube-api-access-n64r2" pod "174e6ae1-f7a9-4ee8-94b9-001dfbb9f387" (UID: "174e6ae1-f7a9-4ee8-94b9-001dfbb9f387"). InnerVolumeSpecName "kube-api-access-n64r2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:55.281766 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.281125    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-metrics-certs" pod "174e6ae1-f7a9-4ee8-94b9-001dfbb9f387" (UID: "174e6ae1-f7a9-4ee8-94b9-001dfbb9f387"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:55.374113 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.374081    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:55.374113 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.374107    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:55.374302 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.374118    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n64r2\" (UniqueName: \"kubernetes.io/projected/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387-kube-api-access-n64r2\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:55.571102 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.571064    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp" event={"ID":"c86c20cc-52fc-4ffb-aa87-46fcc272c457","Type":"ContainerStarted","Data":"2387be425d06d330d7e35f16350aec3452ad362db57738bfca7c11700e5a8be5"}
Oct 30 19:16:55.571289 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.571144    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp" event={"ID":"c86c20cc-52fc-4ffb-aa87-46fcc272c457","Type":"ContainerStarted","Data":"533ab8932b7b13081936d4f3c7d6f50890181759df79775a87ec0a661bcb516a"}
Oct 30 19:16:55.572431 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.572401    5820 generic.go:356] "Generic (PLEG): container finished" podID="174e6ae1-f7a9-4ee8-94b9-001dfbb9f387" containerID="83cb33008e2c921ddbe236313c964af510c77b500ae8c25781421209bda0d5d9" exitCode=0
Oct 30 19:16:55.572557 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.572442    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"
Oct 30 19:16:55.572557 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.572482    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x" event={"ID":"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387","Type":"ContainerDied","Data":"83cb33008e2c921ddbe236313c964af510c77b500ae8c25781421209bda0d5d9"}
Oct 30 19:16:55.572557 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.572532    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x" event={"ID":"174e6ae1-f7a9-4ee8-94b9-001dfbb9f387","Type":"ContainerDied","Data":"f3bf1748577cf75244f62c7653ea9699311901dadb48816215419754f1073202"}
Oct 30 19:16:55.572557 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.572555    5820 scope.go:122] "RemoveContainer" containerID="83cb33008e2c921ddbe236313c964af510c77b500ae8c25781421209bda0d5d9"
Oct 30 19:16:55.582239 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.582219    5820 scope.go:122] "RemoveContainer" containerID="83cb33008e2c921ddbe236313c964af510c77b500ae8c25781421209bda0d5d9"
Oct 30 19:16:55.582537 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:16:55.582519    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83cb33008e2c921ddbe236313c964af510c77b500ae8c25781421209bda0d5d9\": container with ID starting with 83cb33008e2c921ddbe236313c964af510c77b500ae8c25781421209bda0d5d9 not found: ID does not exist" containerID="83cb33008e2c921ddbe236313c964af510c77b500ae8c25781421209bda0d5d9"
Oct 30 19:16:55.582584 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.582545    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83cb33008e2c921ddbe236313c964af510c77b500ae8c25781421209bda0d5d9"} err="failed to get container status \"83cb33008e2c921ddbe236313c964af510c77b500ae8c25781421209bda0d5d9\": rpc error: code = NotFound desc = could not find container \"83cb33008e2c921ddbe236313c964af510c77b500ae8c25781421209bda0d5d9\": container with ID starting with 83cb33008e2c921ddbe236313c964af510c77b500ae8c25781421209bda0d5d9 not found: ID does not exist"
Oct 30 19:16:55.699282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.699228    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp" podStartSLOduration=2.699214456 podStartE2EDuration="2.699214456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:16:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:16:55.697186217 +0000 UTC m=+2536.408934914" watchObservedRunningTime="2026-10-30 19:16:55.699214456 +0000 UTC m=+2536.410963152"
Oct 30 19:16:55.820685 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.820649    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"]
Oct 30 19:16:55.820878 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:55.820709    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f554bb-57bc958746-kth6x"]
Oct 30 19:16:56.165125 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:56.165032    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:16:56.166692 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:56.166664    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 30 19:16:56.166837 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:56.166752    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp" podUID="c86c20cc-52fc-4ffb-aa87-46fcc272c457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 30 19:16:56.602034 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:56.601995    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:56.847006 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:56.841453    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:57.045689 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:57.045595    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:57.165341 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:57.165304    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 30 19:16:57.165730 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:57.165377    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp" podUID="c86c20cc-52fc-4ffb-aa87-46fcc272c457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 30 19:16:57.445974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:57.440583    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="174e6ae1-f7a9-4ee8-94b9-001dfbb9f387" path="/var/lib/kubelet/pods/174e6ae1-f7a9-4ee8-94b9-001dfbb9f387/volumes"
Oct 30 19:16:57.563235 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:57.563195    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:58.168974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:58.168291    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 30 19:16:58.168974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:58.168366    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp" podUID="c86c20cc-52fc-4ffb-aa87-46fcc272c457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 30 19:16:59.164512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:59.164477    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 30 19:16:59.164691 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:16:59.164531    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp" podUID="c86c20cc-52fc-4ffb-aa87-46fcc272c457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 30 19:17:00.165556 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:00.165511    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 30 19:17:00.166036 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:00.165566    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp" podUID="c86c20cc-52fc-4ffb-aa87-46fcc272c457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 30 19:17:00.798896 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:00.798863    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:01.164941 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:01.164857    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 30 19:17:01.164941 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:01.164930    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp" podUID="c86c20cc-52fc-4ffb-aa87-46fcc272c457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 30 19:17:02.289023 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:02.285440    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:02.307245 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:02.302285    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:17:02.307245 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:02.303740    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:17:02.316270 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:02.316210    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:17:04.028332 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:04.028293    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:04.256426 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:04.256384    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:04.440451 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:04.440312    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:08.203862 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:17:08.203826    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:17:08.990358 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:08.990326    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"]
Oct 30 19:17:08.990870 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:08.990857    5820 cpu_manager.go:414] "containerMap: removing container" podUID="4a0996ab-5aaa-4b71-b064-87d9d1572d13" containerName="istio-proxy"
Oct 30 19:17:08.990932 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:08.990871    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a0996ab-5aaa-4b71-b064-87d9d1572d13" containerName="istio-proxy"
Oct 30 19:17:08.990932 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:08.990896    5820 cpu_manager.go:414] "containerMap: removing container" podUID="174e6ae1-f7a9-4ee8-94b9-001dfbb9f387" containerName="maas-api"
Oct 30 19:17:08.990932 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:08.990901    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="174e6ae1-f7a9-4ee8-94b9-001dfbb9f387" containerName="maas-api"
Oct 30 19:17:08.991056 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:08.991013    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a0996ab-5aaa-4b71-b064-87d9d1572d13" containerName="istio-proxy"
Oct 30 19:17:08.991056 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:08.991027    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd708157-c511-4334-84c0-6d0a655e3502" containerName="istio-proxy"
Oct 30 19:17:08.991056 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:08.991034    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="174e6ae1-f7a9-4ee8-94b9-001dfbb9f387" containerName="maas-api"
Oct 30 19:17:08.994317 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:08.994299    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.079591 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.079556    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.079763 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.079620    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.079763 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.079738    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.079890 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.079824    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.079890 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.079876    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.080032 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.079892    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.080032 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.080001    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6r6j\" (UniqueName: \"kubernetes.io/projected/4cbca4a3-6539-477b-a858-5b563ebd3bfb-kube-api-access-q6r6j\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.080032 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.080021    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.080203 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.080078    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.181407 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.181375    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6r6j\" (UniqueName: \"kubernetes.io/projected/4cbca4a3-6539-477b-a858-5b563ebd3bfb-kube-api-access-q6r6j\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.181407 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.181419    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.181660 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.181484    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.181660 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.181575    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.181660 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.181627    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.181827 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.181674    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.181827 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.181754    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.181943 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.181839    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.181943 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.181865    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.182105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.181863    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.182105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.182004    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.182105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.182060    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.182331 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.182303    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.182378 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.182311    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.184267 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.184234    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.184437 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.184419    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.211767 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.211738    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"]
Oct 30 19:17:09.253842 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.252881    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:09.256768 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.256726    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.260998 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.260968    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6r6j\" (UniqueName: \"kubernetes.io/projected/4cbca4a3-6539-477b-a858-5b563ebd3bfb-kube-api-access-q6r6j\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.307894 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.307867    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:09.739601 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.739568    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"]
Oct 30 19:17:09.741116 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:17:09.741088    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cbca4a3_6539_477b_a858_5b563ebd3bfb.slice/crio-d277764d5333faa0b8fd4b3eafde71a2b6372317d8d414d1c5fd534cc5aadce6 WatchSource:0}: Error finding container d277764d5333faa0b8fd4b3eafde71a2b6372317d8d414d1c5fd534cc5aadce6: Status 404 returned error can't find the container with id d277764d5333faa0b8fd4b3eafde71a2b6372317d8d414d1c5fd534cc5aadce6
Oct 30 19:17:09.750873 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.744443    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:17:09.750873 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.744609    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:17:09.750873 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:09.744700    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:17:10.582251 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.582213    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"]
Oct 30 19:17:10.586545 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.586523    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.597479 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.597457    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istiod-ca-cert\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.597602 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.597487    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-credential-socket\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.597602 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.597530    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-workload-socket\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.597746 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.597687    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-token\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.597794 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.597743    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-workload-certs\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.597843 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.597825    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq8bv\" (UniqueName: \"kubernetes.io/projected/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-kube-api-access-sq8bv\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.597921 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.597900    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-data\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.598026 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.598013    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-podinfo\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.598070 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.598050    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-envoy\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.662011 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.660208    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-4a7a52-openshift-default-dockercfg-9p592\""
Oct 30 19:17:10.662011 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.660257    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"]
Oct 30 19:17:10.669359 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.669322    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd" event={"ID":"4cbca4a3-6539-477b-a858-5b563ebd3bfb","Type":"ContainerStarted","Data":"ec1868e380d4afc51c4ffdb19c55ea43135433bc167c99ddbce69d6a376618c6"}
Oct 30 19:17:10.669513 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.669385    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd" event={"ID":"4cbca4a3-6539-477b-a858-5b563ebd3bfb","Type":"ContainerStarted","Data":"d277764d5333faa0b8fd4b3eafde71a2b6372317d8d414d1c5fd534cc5aadce6"}
Oct 30 19:17:10.699745 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.699711    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-envoy\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.699745 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.699782    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istiod-ca-cert\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.699745 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.699804    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-credential-socket\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.700230 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.700010    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-workload-socket\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.700230 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.700062    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-token\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.700230 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.700116    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-workload-certs\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.700230 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.700163    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq8bv\" (UniqueName: \"kubernetes.io/projected/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-kube-api-access-sq8bv\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.700230 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.700229    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-data\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.700473 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.700357    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-podinfo\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.700880 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.700854    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-workload-certs\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.701068 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.700882    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-workload-socket\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.701162 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.700965    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istiod-ca-cert\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.701339 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.700980    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-data\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.701450 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.701154    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-credential-socket\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.703063 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.703042    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-podinfo\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.703750 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.703731    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-envoy\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.763233 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.763196    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-token\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.771208 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.771177    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq8bv\" (UniqueName: \"kubernetes.io/projected/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-kube-api-access-sq8bv\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:10.860263 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.860132    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd" podStartSLOduration=2.860108947 podStartE2EDuration="2.860108947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:17:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:17:10.849447612 +0000 UTC m=+2551.561196314" watchObservedRunningTime="2026-10-30 19:17:10.860108947 +0000 UTC m=+2551.571857645"
Oct 30 19:17:10.900867 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:10.900836    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:11.308515 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:11.308481    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:11.310280 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:11.310258    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 30 19:17:11.310339 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:11.310305    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd" podUID="4cbca4a3-6539-477b-a858-5b563ebd3bfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 30 19:17:11.361359 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:11.361329    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"]
Oct 30 19:17:11.398584 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:17:11.398555    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c64155a_21e5_4b7c_99f9_f2d35fc457b0.slice/crio-a972e9f6c8429f29bba7259bf0c2647e811f97dd52923dec7e1c43273ea699c2 WatchSource:0}: Error finding container a972e9f6c8429f29bba7259bf0c2647e811f97dd52923dec7e1c43273ea699c2: Status 404 returned error can't find the container with id a972e9f6c8429f29bba7259bf0c2647e811f97dd52923dec7e1c43273ea699c2
Oct 30 19:17:11.401232 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:11.401199    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:17:11.401347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:11.401277    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:17:11.401347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:11.401321    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:17:11.677248 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:11.677133    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm" event={"ID":"2c64155a-21e5-4b7c-99f9-f2d35fc457b0","Type":"ContainerStarted","Data":"56344f47feeeb1209c55b56f1b0503ab319b3eb7c0053019230e6fad150e1841"}
Oct 30 19:17:11.677248 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:11.677214    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm" event={"ID":"2c64155a-21e5-4b7c-99f9-f2d35fc457b0","Type":"ContainerStarted","Data":"a972e9f6c8429f29bba7259bf0c2647e811f97dd52923dec7e1c43273ea699c2"}
Oct 30 19:17:11.799067 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:11.799004    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm" podStartSLOduration=1.7989911140000001 podStartE2EDuration="1.798991114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:17:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:17:11.796843645 +0000 UTC m=+2552.508592343" watchObservedRunningTime="2026-10-30 19:17:11.798991114 +0000 UTC m=+2552.510739823"
Oct 30 19:17:11.901817 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:11.901784    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:12.308671 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:12.308621    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 30 19:17:12.308840 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:12.308683    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd" podUID="4cbca4a3-6539-477b-a858-5b563ebd3bfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 30 19:17:12.903076 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:12.903037    5820 patch_prober.go:29] interesting pod/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:17:12.903502 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:12.903085    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm" podUID="2c64155a-21e5-4b7c-99f9-f2d35fc457b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 19:17:13.309385 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:13.309350    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 30 19:17:13.309589 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:13.309401    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd" podUID="4cbca4a3-6539-477b-a858-5b563ebd3bfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 30 19:17:13.519127 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:13.519085    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"]
Oct 30 19:17:13.519692 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:13.519568    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4" podUID="6d71958e-32bc-43db-af6d-157ebd0af8eb" containerName="maas-api" containerID="cri-o://83fd5e125a51561273ef4d6289392f1389ee8c3bf747b3e4bcc89ba04cea823f" gracePeriod=30
Oct 30 19:17:13.709060 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:13.688918    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:13.814055 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:13.813934    5820 generic.go:356] "Generic (PLEG): container finished" podID="6d71958e-32bc-43db-af6d-157ebd0af8eb" containerID="83fd5e125a51561273ef4d6289392f1389ee8c3bf747b3e4bcc89ba04cea823f" exitCode=0
Oct 30 19:17:13.818003 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:13.814527    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4" event={"ID":"6d71958e-32bc-43db-af6d-157ebd0af8eb","Type":"ContainerDied","Data":"83fd5e125a51561273ef4d6289392f1389ee8c3bf747b3e4bcc89ba04cea823f"}
Oct 30 19:17:13.990852 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:13.921194    5820 patch_prober.go:29] interesting pod/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:17:13.990852 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:13.921253    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm" podUID="2c64155a-21e5-4b7c-99f9-f2d35fc457b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 19:17:14.106656 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:14.106624    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"
Oct 30 19:17:14.122639 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:14.122612    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6d71958e-32bc-43db-af6d-157ebd0af8eb-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d71958e-32bc-43db-af6d-157ebd0af8eb-maas-api-tls\") pod \"6d71958e-32bc-43db-af6d-157ebd0af8eb\" (UID: \"6d71958e-32bc-43db-af6d-157ebd0af8eb\") "
Oct 30 19:17:14.122849 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:14.122832    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6d71958e-32bc-43db-af6d-157ebd0af8eb-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d71958e-32bc-43db-af6d-157ebd0af8eb-openshift-service-ca\") pod \"6d71958e-32bc-43db-af6d-157ebd0af8eb\" (UID: \"6d71958e-32bc-43db-af6d-157ebd0af8eb\") "
Oct 30 19:17:14.123016 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:14.122999    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d71958e-32bc-43db-af6d-157ebd0af8eb-kube-api-access-74rsr\" (UniqueName: \"kubernetes.io/projected/6d71958e-32bc-43db-af6d-157ebd0af8eb-kube-api-access-74rsr\") pod \"6d71958e-32bc-43db-af6d-157ebd0af8eb\" (UID: \"6d71958e-32bc-43db-af6d-157ebd0af8eb\") "
Oct 30 19:17:14.123181 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:14.123165    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6d71958e-32bc-43db-af6d-157ebd0af8eb-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d71958e-32bc-43db-af6d-157ebd0af8eb-metrics-certs\") pod \"6d71958e-32bc-43db-af6d-157ebd0af8eb\" (UID: \"6d71958e-32bc-43db-af6d-157ebd0af8eb\") "
Oct 30 19:17:14.123473 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:14.123219    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d71958e-32bc-43db-af6d-157ebd0af8eb-openshift-service-ca" pod "6d71958e-32bc-43db-af6d-157ebd0af8eb" (UID: "6d71958e-32bc-43db-af6d-157ebd0af8eb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:14.124038 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:14.123897    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d71958e-32bc-43db-af6d-157ebd0af8eb-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:14.125349 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:14.125323    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d71958e-32bc-43db-af6d-157ebd0af8eb-maas-api-tls" pod "6d71958e-32bc-43db-af6d-157ebd0af8eb" (UID: "6d71958e-32bc-43db-af6d-157ebd0af8eb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:17:14.127259 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:14.127235    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d71958e-32bc-43db-af6d-157ebd0af8eb-metrics-certs" pod "6d71958e-32bc-43db-af6d-157ebd0af8eb" (UID: "6d71958e-32bc-43db-af6d-157ebd0af8eb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:17:14.130311 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:14.130288    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d71958e-32bc-43db-af6d-157ebd0af8eb-kube-api-access-74rsr" pod "6d71958e-32bc-43db-af6d-157ebd0af8eb" (UID: "6d71958e-32bc-43db-af6d-157ebd0af8eb"). InnerVolumeSpecName "kube-api-access-74rsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:14.194975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:14.193722    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:14.224852 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:14.224815    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d71958e-32bc-43db-af6d-157ebd0af8eb-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:14.224852 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:14.224847    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74rsr\" (UniqueName: \"kubernetes.io/projected/6d71958e-32bc-43db-af6d-157ebd0af8eb-kube-api-access-74rsr\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:14.224852 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:14.224860    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d71958e-32bc-43db-af6d-157ebd0af8eb-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:14.309165 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:14.309128    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 30 19:17:14.309339 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:14.309177    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd" podUID="4cbca4a3-6539-477b-a858-5b563ebd3bfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 30 19:17:14.386920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:14.386648    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:14.821442 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:14.821403    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4" event={"ID":"6d71958e-32bc-43db-af6d-157ebd0af8eb","Type":"ContainerDied","Data":"6fb59fdee89a0642c0b5e6e4c8e227bec7dbc8dfd1701b57d62bbe11f9f21d37"}
Oct 30 19:17:14.821648 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:14.821471    5820 scope.go:122] "RemoveContainer" containerID="83fd5e125a51561273ef4d6289392f1389ee8c3bf747b3e4bcc89ba04cea823f"
Oct 30 19:17:14.821648 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:14.821619    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"
Oct 30 19:17:14.901237 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:14.901208    5820 patch_prober.go:29] interesting pod/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:17:14.901426 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:14.901256    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm" podUID="2c64155a-21e5-4b7c-99f9-f2d35fc457b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 19:17:15.015359 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:15.015322    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"]
Oct 30 19:17:15.019710 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:15.019679    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f44c7-64bccf6b48-jghl4"]
Oct 30 19:17:15.225194 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:15.225114    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:15.308553 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:15.308511    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 30 19:17:15.308749 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:15.308564    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd" podUID="4cbca4a3-6539-477b-a858-5b563ebd3bfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 30 19:17:15.449976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:15.445926    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6d71958e-32bc-43db-af6d-157ebd0af8eb" path="/var/lib/kubelet/pods/6d71958e-32bc-43db-af6d-157ebd0af8eb/volumes"
Oct 30 19:17:15.503550 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:15.503471    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:15.808970 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:15.804872    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:15.901174 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:15.901145    5820 patch_prober.go:29] interesting pod/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:17:15.901354 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:15.901191    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm" podUID="2c64155a-21e5-4b7c-99f9-f2d35fc457b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 19:17:16.179181 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:16.179088    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:16.308662 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:16.308625    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 30 19:17:16.308840 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:16.308686    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd" podUID="4cbca4a3-6539-477b-a858-5b563ebd3bfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 30 19:17:16.901426 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:16.901393    5820 patch_prober.go:29] interesting pod/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:17:16.901594 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:16.901447    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm" podUID="2c64155a-21e5-4b7c-99f9-f2d35fc457b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 19:17:17.349305 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:17.349271    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:17.351634 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:17.351613    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:17.359970 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:17.356176    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:17:17.902158 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:17.902125    5820 patch_prober.go:29] interesting pod/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:17:17.902310 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:17.902180    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm" podUID="2c64155a-21e5-4b7c-99f9-f2d35fc457b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 19:17:17.952611 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:17.952577    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:18.139205 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:18.139164    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:18.938182 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:18.938150    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:18.938756 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:18.938735    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:18.939620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:18.939588    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:17:19.666778 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:19.666734    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:20.139050 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:20.139015    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:22.733346 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:22.733208    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:22.956391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:22.956350    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:23.210810 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:23.210779    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"]
Oct 30 19:17:23.211020 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:23.210994    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t" podUID="223cde4d-b792-432f-80ca-fb0189521a71" containerName="istio-proxy" containerID="cri-o://d11e3e38d1e67d79be444ea8cf05e3d0f6f1edcc9fbf4f64cb8395cb7444a413" gracePeriod=30
Oct 30 19:17:23.255737 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:23.255700    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:24.887285 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:24.887248    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:25.341573 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:25.341537    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:25.709391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:25.709301    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:26.005733 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:26.005647    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:27.733568 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:27.733532    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:28.049294 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.049261    5820 patch_prober.go:29] interesting pod/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:17:28.049463 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.049321    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t" podUID="223cde4d-b792-432f-80ca-fb0189521a71" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:17:28.465236 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.465208    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:17:28.545541 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.545514    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-istio-data\") pod \"223cde4d-b792-432f-80ca-fb0189521a71\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") "
Oct 30 19:17:28.545682 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.545555    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-istio-envoy\") pod \"223cde4d-b792-432f-80ca-fb0189521a71\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") "
Oct 30 19:17:28.545682 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.545606    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-workload-certs\") pod \"223cde4d-b792-432f-80ca-fb0189521a71\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") "
Oct 30 19:17:28.545802 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.545698    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-credential-socket\") pod \"223cde4d-b792-432f-80ca-fb0189521a71\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") "
Oct 30 19:17:28.545802 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.545736    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-istio-data" pod "223cde4d-b792-432f-80ca-fb0189521a71" (UID: "223cde4d-b792-432f-80ca-fb0189521a71"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:28.545802 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.545768    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/223cde4d-b792-432f-80ca-fb0189521a71-kube-api-access-kt2dv\" (UniqueName: \"kubernetes.io/projected/223cde4d-b792-432f-80ca-fb0189521a71-kube-api-access-kt2dv\") pod \"223cde4d-b792-432f-80ca-fb0189521a71\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") "
Oct 30 19:17:28.546000 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.545801    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/223cde4d-b792-432f-80ca-fb0189521a71-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/223cde4d-b792-432f-80ca-fb0189521a71-istio-podinfo\") pod \"223cde4d-b792-432f-80ca-fb0189521a71\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") "
Oct 30 19:17:28.546000 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.545891    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-workload-socket\") pod \"223cde4d-b792-432f-80ca-fb0189521a71\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") "
Oct 30 19:17:28.546000 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.545928    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/223cde4d-b792-432f-80ca-fb0189521a71-istio-token\" (UniqueName: \"kubernetes.io/projected/223cde4d-b792-432f-80ca-fb0189521a71-istio-token\") pod \"223cde4d-b792-432f-80ca-fb0189521a71\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") "
Oct 30 19:17:28.546000 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.545976    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-workload-certs" pod "223cde4d-b792-432f-80ca-fb0189521a71" (UID: "223cde4d-b792-432f-80ca-fb0189521a71"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:28.546225 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.546008    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/223cde4d-b792-432f-80ca-fb0189521a71-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/223cde4d-b792-432f-80ca-fb0189521a71-istiod-ca-cert\") pod \"223cde4d-b792-432f-80ca-fb0189521a71\" (UID: \"223cde4d-b792-432f-80ca-fb0189521a71\") "
Oct 30 19:17:28.546225 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.546076    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-credential-socket" pod "223cde4d-b792-432f-80ca-fb0189521a71" (UID: "223cde4d-b792-432f-80ca-fb0189521a71"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:28.546329 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.546223    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-workload-socket" pod "223cde4d-b792-432f-80ca-fb0189521a71" (UID: "223cde4d-b792-432f-80ca-fb0189521a71"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:28.546591 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.546553    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/223cde4d-b792-432f-80ca-fb0189521a71-istiod-ca-cert" pod "223cde4d-b792-432f-80ca-fb0189521a71" (UID: "223cde4d-b792-432f-80ca-fb0189521a71"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:28.546868 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.546619    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:28.546868 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.546635    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:28.546868 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.546647    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:28.546868 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.546656    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:28.547984 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.547918    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-istio-envoy" pod "223cde4d-b792-432f-80ca-fb0189521a71" (UID: "223cde4d-b792-432f-80ca-fb0189521a71"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:28.548330 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.548310    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/223cde4d-b792-432f-80ca-fb0189521a71-istio-token" pod "223cde4d-b792-432f-80ca-fb0189521a71" (UID: "223cde4d-b792-432f-80ca-fb0189521a71"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:28.548502 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.548481    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/223cde4d-b792-432f-80ca-fb0189521a71-istio-podinfo" pod "223cde4d-b792-432f-80ca-fb0189521a71" (UID: "223cde4d-b792-432f-80ca-fb0189521a71"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:28.548597 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.548512    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/223cde4d-b792-432f-80ca-fb0189521a71-kube-api-access-kt2dv" pod "223cde4d-b792-432f-80ca-fb0189521a71" (UID: "223cde4d-b792-432f-80ca-fb0189521a71"). InnerVolumeSpecName "kube-api-access-kt2dv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:28.648114 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.648032    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/223cde4d-b792-432f-80ca-fb0189521a71-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:28.648114 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.648056    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kt2dv\" (UniqueName: \"kubernetes.io/projected/223cde4d-b792-432f-80ca-fb0189521a71-kube-api-access-kt2dv\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:28.648114 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.648082    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/223cde4d-b792-432f-80ca-fb0189521a71-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:28.648114 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.648091    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/223cde4d-b792-432f-80ca-fb0189521a71-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:28.648114 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.648099    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/223cde4d-b792-432f-80ca-fb0189521a71-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:28.935543 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.935427    5820 generic.go:356] "Generic (PLEG): container finished" podID="223cde4d-b792-432f-80ca-fb0189521a71" containerID="d11e3e38d1e67d79be444ea8cf05e3d0f6f1edcc9fbf4f64cb8395cb7444a413" exitCode=0
Oct 30 19:17:28.935543 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.935509    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t" event={"ID":"223cde4d-b792-432f-80ca-fb0189521a71","Type":"ContainerDied","Data":"d11e3e38d1e67d79be444ea8cf05e3d0f6f1edcc9fbf4f64cb8395cb7444a413"}
Oct 30 19:17:28.936164 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.935615    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t" event={"ID":"223cde4d-b792-432f-80ca-fb0189521a71","Type":"ContainerDied","Data":"abe9ba0dd60a33cbb3d59a0fdfe7e76256f4aa0950518a2f089fb44d8b1e355b"}
Oct 30 19:17:28.936164 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.935656    5820 scope.go:122] "RemoveContainer" containerID="d11e3e38d1e67d79be444ea8cf05e3d0f6f1edcc9fbf4f64cb8395cb7444a413"
Oct 30 19:17:28.936164 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.935853    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"
Oct 30 19:17:28.955138 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.955106    5820 scope.go:122] "RemoveContainer" containerID="60d15b8bdd9eb7c0a539a8d15b12e2b971d4d6f31191180766b0eee8eed16b30"
Oct 30 19:17:28.964349 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.964325    5820 scope.go:122] "RemoveContainer" containerID="d11e3e38d1e67d79be444ea8cf05e3d0f6f1edcc9fbf4f64cb8395cb7444a413"
Oct 30 19:17:28.964631 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:17:28.964609    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d11e3e38d1e67d79be444ea8cf05e3d0f6f1edcc9fbf4f64cb8395cb7444a413\": container with ID starting with d11e3e38d1e67d79be444ea8cf05e3d0f6f1edcc9fbf4f64cb8395cb7444a413 not found: ID does not exist" containerID="d11e3e38d1e67d79be444ea8cf05e3d0f6f1edcc9fbf4f64cb8395cb7444a413"
Oct 30 19:17:28.964707 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.964640    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d11e3e38d1e67d79be444ea8cf05e3d0f6f1edcc9fbf4f64cb8395cb7444a413"} err="failed to get container status \"d11e3e38d1e67d79be444ea8cf05e3d0f6f1edcc9fbf4f64cb8395cb7444a413\": rpc error: code = NotFound desc = could not find container \"d11e3e38d1e67d79be444ea8cf05e3d0f6f1edcc9fbf4f64cb8395cb7444a413\": container with ID starting with d11e3e38d1e67d79be444ea8cf05e3d0f6f1edcc9fbf4f64cb8395cb7444a413 not found: ID does not exist"
Oct 30 19:17:28.964707 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.964658    5820 scope.go:122] "RemoveContainer" containerID="60d15b8bdd9eb7c0a539a8d15b12e2b971d4d6f31191180766b0eee8eed16b30"
Oct 30 19:17:28.964919 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:17:28.964900    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60d15b8bdd9eb7c0a539a8d15b12e2b971d4d6f31191180766b0eee8eed16b30\": container with ID starting with 60d15b8bdd9eb7c0a539a8d15b12e2b971d4d6f31191180766b0eee8eed16b30 not found: ID does not exist" containerID="60d15b8bdd9eb7c0a539a8d15b12e2b971d4d6f31191180766b0eee8eed16b30"
Oct 30 19:17:28.965010 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:28.964923    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60d15b8bdd9eb7c0a539a8d15b12e2b971d4d6f31191180766b0eee8eed16b30"} err="failed to get container status \"60d15b8bdd9eb7c0a539a8d15b12e2b971d4d6f31191180766b0eee8eed16b30\": rpc error: code = NotFound desc = could not find container \"60d15b8bdd9eb7c0a539a8d15b12e2b971d4d6f31191180766b0eee8eed16b30\": container with ID starting with 60d15b8bdd9eb7c0a539a8d15b12e2b971d4d6f31191180766b0eee8eed16b30 not found: ID does not exist"
Oct 30 19:17:29.147804 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:29.147770    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"]
Oct 30 19:17:29.166055 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:29.166027    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-7f44c7-openshift-default-54f6769b4c-d9f7t"]
Oct 30 19:17:29.436764 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:29.436734    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="223cde4d-b792-432f-80ca-fb0189521a71" path="/var/lib/kubelet/pods/223cde4d-b792-432f-80ca-fb0189521a71/volumes"
Oct 30 19:17:29.677721 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:29.677684    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:29.897444 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:29.897412    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:30.146065 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:30.146030    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:32.915130 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:32.915094    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:33.129273 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.129236    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:33.482974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.475232    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z"]
Oct 30 19:17:33.482974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.476063    5820 cpu_manager.go:414] "containerMap: removing container" podUID="6d71958e-32bc-43db-af6d-157ebd0af8eb" containerName="maas-api"
Oct 30 19:17:33.482974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.476083    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6d71958e-32bc-43db-af6d-157ebd0af8eb" containerName="maas-api"
Oct 30 19:17:33.482974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.476112    5820 cpu_manager.go:414] "containerMap: removing container" podUID="223cde4d-b792-432f-80ca-fb0189521a71" containerName="istio-proxy"
Oct 30 19:17:33.482974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.476164    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="223cde4d-b792-432f-80ca-fb0189521a71" containerName="istio-proxy"
Oct 30 19:17:33.482974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.476181    5820 cpu_manager.go:414] "containerMap: removing container" podUID="223cde4d-b792-432f-80ca-fb0189521a71" containerName="istio-proxy"
Oct 30 19:17:33.482974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.476189    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="223cde4d-b792-432f-80ca-fb0189521a71" containerName="istio-proxy"
Oct 30 19:17:33.482974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.476374    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="6d71958e-32bc-43db-af6d-157ebd0af8eb" containerName="maas-api"
Oct 30 19:17:33.482974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.476399    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="223cde4d-b792-432f-80ca-fb0189521a71" containerName="istio-proxy"
Oct 30 19:17:33.482974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.476410    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="223cde4d-b792-432f-80ca-fb0189521a71" containerName="istio-proxy"
Oct 30 19:17:33.482974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.480454    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z"
Oct 30 19:17:33.504761 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.504705    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z"]
Oct 30 19:17:33.542583 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.542551    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-a-6d953c-dockercfg-s4mk5\""
Oct 30 19:17:33.542583 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.542573    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-a-6d953c\""
Oct 30 19:17:33.589004 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.588969    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg"]
Oct 30 19:17:33.592890 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.592874    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg"
Oct 30 19:17:33.619919 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.619894    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tkbr\" (UniqueName: \"kubernetes.io/projected/e89efa10-f7ae-49b3-93b9-a939802f3f4e-kube-api-access-2tkbr\") pod \"payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z\" (UID: \"e89efa10-f7ae-49b3-93b9-a939802f3f4e\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z"
Oct 30 19:17:33.620077 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.620042    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e89efa10-f7ae-49b3-93b9-a939802f3f4e-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z\" (UID: \"e89efa10-f7ae-49b3-93b9-a939802f3f4e\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z"
Oct 30 19:17:33.650791 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.650759    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg"]
Oct 30 19:17:33.721256 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.721228    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tkbr\" (UniqueName: \"kubernetes.io/projected/e89efa10-f7ae-49b3-93b9-a939802f3f4e-kube-api-access-2tkbr\") pod \"payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z\" (UID: \"e89efa10-f7ae-49b3-93b9-a939802f3f4e\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z"
Oct 30 19:17:33.721416 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.721340    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e89efa10-f7ae-49b3-93b9-a939802f3f4e-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z\" (UID: \"e89efa10-f7ae-49b3-93b9-a939802f3f4e\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z"
Oct 30 19:17:33.721487 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.721432    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f2b22b3-ad10-455d-a648-2100f39e3232-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg\" (UID: \"2f2b22b3-ad10-455d-a648-2100f39e3232\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg"
Oct 30 19:17:33.721487 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.721465    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd7xj\" (UniqueName: \"kubernetes.io/projected/2f2b22b3-ad10-455d-a648-2100f39e3232-kube-api-access-gd7xj\") pod \"payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg\" (UID: \"2f2b22b3-ad10-455d-a648-2100f39e3232\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg"
Oct 30 19:17:33.722703 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.722663    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e89efa10-f7ae-49b3-93b9-a939802f3f4e-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z\" (UID: \"e89efa10-f7ae-49b3-93b9-a939802f3f4e\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z"
Oct 30 19:17:33.782637 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.782565    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tkbr\" (UniqueName: \"kubernetes.io/projected/e89efa10-f7ae-49b3-93b9-a939802f3f4e-kube-api-access-2tkbr\") pod \"payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z\" (UID: \"e89efa10-f7ae-49b3-93b9-a939802f3f4e\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z"
Oct 30 19:17:33.805598 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.805576    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z"
Oct 30 19:17:33.823309 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.823281    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f2b22b3-ad10-455d-a648-2100f39e3232-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg\" (UID: \"2f2b22b3-ad10-455d-a648-2100f39e3232\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg"
Oct 30 19:17:33.823436 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.823325    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd7xj\" (UniqueName: \"kubernetes.io/projected/2f2b22b3-ad10-455d-a648-2100f39e3232-kube-api-access-gd7xj\") pod \"payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg\" (UID: \"2f2b22b3-ad10-455d-a648-2100f39e3232\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg"
Oct 30 19:17:33.824126 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.824101    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f2b22b3-ad10-455d-a648-2100f39e3232-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg\" (UID: \"2f2b22b3-ad10-455d-a648-2100f39e3232\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg"
Oct 30 19:17:33.885186 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.885151    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd7xj\" (UniqueName: \"kubernetes.io/projected/2f2b22b3-ad10-455d-a648-2100f39e3232-kube-api-access-gd7xj\") pod \"payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg\" (UID: \"2f2b22b3-ad10-455d-a648-2100f39e3232\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg"
Oct 30 19:17:33.906893 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:33.906867    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg"
Oct 30 19:17:34.181422 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:34.181397    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z"]
Oct 30 19:17:34.183175 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:17:34.183147    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode89efa10_f7ae_49b3_93b9_a939802f3f4e.slice/crio-3b81823657ba83815c091a883e6144b3adafc3537e68ebaf51ecf92d1d5cb83b WatchSource:0}: Error finding container 3b81823657ba83815c091a883e6144b3adafc3537e68ebaf51ecf92d1d5cb83b: Status 404 returned error can't find the container with id 3b81823657ba83815c091a883e6144b3adafc3537e68ebaf51ecf92d1d5cb83b
Oct 30 19:17:34.284348 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:34.284324    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg"]
Oct 30 19:17:34.285897 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:17:34.285862    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f2b22b3_ad10_455d_a648_2100f39e3232.slice/crio-a05fe0bb4c0eba6264eb0e0a903f1925853520ab7162cb5089c0b5a269eb0f9d WatchSource:0}: Error finding container a05fe0bb4c0eba6264eb0e0a903f1925853520ab7162cb5089c0b5a269eb0f9d: Status 404 returned error can't find the container with id a05fe0bb4c0eba6264eb0e0a903f1925853520ab7162cb5089c0b5a269eb0f9d
Oct 30 19:17:35.040083 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:35.040043    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z" event={"ID":"e89efa10-f7ae-49b3-93b9-a939802f3f4e","Type":"ContainerStarted","Data":"762eea0d0102a81b6f400d3de4a3c5894a5867c3ad7784e3865cf9862b370b8e"}
Oct 30 19:17:35.040295 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:35.040099    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z" event={"ID":"e89efa10-f7ae-49b3-93b9-a939802f3f4e","Type":"ContainerStarted","Data":"3b81823657ba83815c091a883e6144b3adafc3537e68ebaf51ecf92d1d5cb83b"}
Oct 30 19:17:35.040295 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:35.040138    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z"
Oct 30 19:17:35.041875 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:35.041851    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg" event={"ID":"2f2b22b3-ad10-455d-a648-2100f39e3232","Type":"ContainerStarted","Data":"ecfe566adf1238afaca927ec6dadb6771e8c84e4f7788cb00d17c4c2d2d8bf37"}
Oct 30 19:17:35.042003 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:35.041890    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg" event={"ID":"2f2b22b3-ad10-455d-a648-2100f39e3232","Type":"ContainerStarted","Data":"a05fe0bb4c0eba6264eb0e0a903f1925853520ab7162cb5089c0b5a269eb0f9d"}
Oct 30 19:17:35.042003 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:35.041938    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg"
Oct 30 19:17:35.157144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:35.157099    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z" podStartSLOduration=2.157083499 podStartE2EDuration="2.157083499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:17:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:17:35.15580221 +0000 UTC m=+2575.867550909" watchObservedRunningTime="2026-10-30 19:17:35.157083499 +0000 UTC m=+2575.868832197"
Oct 30 19:17:35.291515 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:35.290225    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg" podStartSLOduration=2.290200387 podStartE2EDuration="2.290200387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:17:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:17:35.278814389 +0000 UTC m=+2575.990563103" watchObservedRunningTime="2026-10-30 19:17:35.290200387 +0000 UTC m=+2576.001949112"
Oct 30 19:17:38.146098 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:38.146062    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:38.203841 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:17:38.203806    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:17:38.306985 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:38.306939    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"]
Oct 30 19:17:38.307255 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:38.307223    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz" podUID="ce7148f2-3098-45df-bf88-4edfc8117df0" containerName="istio-proxy" containerID="cri-o://629aed487806c57d49941e388cfb49277090de230e2f220c37c31969e494d85d" gracePeriod=30
Oct 30 19:17:39.626150 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:39.626108    5820 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:17:39.626635 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:39.626184    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz" podUID="ce7148f2-3098-45df-bf88-4edfc8117df0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:17:43.436420 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.436368    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:43.590095 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.590070    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 19:17:43.736756 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.736721    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ce7148f2-3098-45df-bf88-4edfc8117df0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce7148f2-3098-45df-bf88-4edfc8117df0-istiod-ca-cert\") pod \"ce7148f2-3098-45df-bf88-4edfc8117df0\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") "
Oct 30 19:17:43.736756 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.736762    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-workload-certs\") pod \"ce7148f2-3098-45df-bf88-4edfc8117df0\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") "
Oct 30 19:17:43.737014 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.736815    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-envoy\") pod \"ce7148f2-3098-45df-bf88-4edfc8117df0\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") "
Oct 30 19:17:43.737014 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.736852    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce7148f2-3098-45df-bf88-4edfc8117df0-kube-api-access-mm57m\" (UniqueName: \"kubernetes.io/projected/ce7148f2-3098-45df-bf88-4edfc8117df0-kube-api-access-mm57m\") pod \"ce7148f2-3098-45df-bf88-4edfc8117df0\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") "
Oct 30 19:17:43.737014 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.736892    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-token\" (UniqueName: \"kubernetes.io/projected/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-token\") pod \"ce7148f2-3098-45df-bf88-4edfc8117df0\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") "
Oct 30 19:17:43.737014 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.736932    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-credential-socket\") pod \"ce7148f2-3098-45df-bf88-4edfc8117df0\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") "
Oct 30 19:17:43.737234 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.737090    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-data\") pod \"ce7148f2-3098-45df-bf88-4edfc8117df0\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") "
Oct 30 19:17:43.737234 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.737166    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-podinfo\") pod \"ce7148f2-3098-45df-bf88-4edfc8117df0\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") "
Oct 30 19:17:43.737234 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.737177    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce7148f2-3098-45df-bf88-4edfc8117df0-istiod-ca-cert" pod "ce7148f2-3098-45df-bf88-4edfc8117df0" (UID: "ce7148f2-3098-45df-bf88-4edfc8117df0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:43.737234 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.737197    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-workload-certs" pod "ce7148f2-3098-45df-bf88-4edfc8117df0" (UID: "ce7148f2-3098-45df-bf88-4edfc8117df0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:43.737234 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.737208    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-workload-socket\") pod \"ce7148f2-3098-45df-bf88-4edfc8117df0\" (UID: \"ce7148f2-3098-45df-bf88-4edfc8117df0\") "
Oct 30 19:17:43.737496 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.737426    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-credential-socket" pod "ce7148f2-3098-45df-bf88-4edfc8117df0" (UID: "ce7148f2-3098-45df-bf88-4edfc8117df0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:43.737496 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.737435    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-data" pod "ce7148f2-3098-45df-bf88-4edfc8117df0" (UID: "ce7148f2-3098-45df-bf88-4edfc8117df0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:43.737643 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.737626    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-workload-socket" pod "ce7148f2-3098-45df-bf88-4edfc8117df0" (UID: "ce7148f2-3098-45df-bf88-4edfc8117df0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:43.737772 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.737752    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:43.737889 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.737798    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:43.737889 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.737813    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce7148f2-3098-45df-bf88-4edfc8117df0-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:43.737889 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.737826    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:43.737889 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.737840    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:43.739991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.739925    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-envoy" pod "ce7148f2-3098-45df-bf88-4edfc8117df0" (UID: "ce7148f2-3098-45df-bf88-4edfc8117df0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:43.739991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.739976    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce7148f2-3098-45df-bf88-4edfc8117df0-kube-api-access-mm57m" pod "ce7148f2-3098-45df-bf88-4edfc8117df0" (UID: "ce7148f2-3098-45df-bf88-4edfc8117df0"). InnerVolumeSpecName "kube-api-access-mm57m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:43.740140 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.740023    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-podinfo" pod "ce7148f2-3098-45df-bf88-4edfc8117df0" (UID: "ce7148f2-3098-45df-bf88-4edfc8117df0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:43.740140 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.740059    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-token" pod "ce7148f2-3098-45df-bf88-4edfc8117df0" (UID: "ce7148f2-3098-45df-bf88-4edfc8117df0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:43.839527 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.839433    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:43.839527 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.839466    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mm57m\" (UniqueName: \"kubernetes.io/projected/ce7148f2-3098-45df-bf88-4edfc8117df0-kube-api-access-mm57m\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:43.839527 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.839481    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:43.839527 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:43.839493    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce7148f2-3098-45df-bf88-4edfc8117df0-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:44.110673 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:44.110576    5820 generic.go:356] "Generic (PLEG): container finished" podID="ce7148f2-3098-45df-bf88-4edfc8117df0" containerID="629aed487806c57d49941e388cfb49277090de230e2f220c37c31969e494d85d" exitCode=0
Oct 30 19:17:44.110673 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:44.110628    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz" event={"ID":"ce7148f2-3098-45df-bf88-4edfc8117df0","Type":"ContainerDied","Data":"629aed487806c57d49941e388cfb49277090de230e2f220c37c31969e494d85d"}
Oct 30 19:17:44.110673 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:44.110638    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"
Oct 30 19:17:44.110673 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:44.110671    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz" event={"ID":"ce7148f2-3098-45df-bf88-4edfc8117df0","Type":"ContainerDied","Data":"7e69ed11914b4e9446412dcbdc3eec69fb5598334c37e4c4712540343d3eb476"}
Oct 30 19:17:44.111049 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:44.110692    5820 scope.go:122] "RemoveContainer" containerID="629aed487806c57d49941e388cfb49277090de230e2f220c37c31969e494d85d"
Oct 30 19:17:44.121013 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:44.120992    5820 scope.go:122] "RemoveContainer" containerID="629aed487806c57d49941e388cfb49277090de230e2f220c37c31969e494d85d"
Oct 30 19:17:44.121278 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:17:44.121262    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"629aed487806c57d49941e388cfb49277090de230e2f220c37c31969e494d85d\": container with ID starting with 629aed487806c57d49941e388cfb49277090de230e2f220c37c31969e494d85d not found: ID does not exist" containerID="629aed487806c57d49941e388cfb49277090de230e2f220c37c31969e494d85d"
Oct 30 19:17:44.121316 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:44.121286    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"629aed487806c57d49941e388cfb49277090de230e2f220c37c31969e494d85d"} err="failed to get container status \"629aed487806c57d49941e388cfb49277090de230e2f220c37c31969e494d85d\": rpc error: code = NotFound desc = could not find container \"629aed487806c57d49941e388cfb49277090de230e2f220c37c31969e494d85d\": container with ID starting with 629aed487806c57d49941e388cfb49277090de230e2f220c37c31969e494d85d not found: ID does not exist"
Oct 30 19:17:44.275697 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:44.275649    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"]
Oct 30 19:17:44.324883 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:44.324853    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-2vjmz"]
Oct 30 19:17:45.436761 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:45.436723    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce7148f2-3098-45df-bf88-4edfc8117df0" path="/var/lib/kubelet/pods/ce7148f2-3098-45df-bf88-4edfc8117df0/volumes"
Oct 30 19:17:46.051660 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:46.051631    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg"
Oct 30 19:17:46.051838 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:46.051793    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z"
Oct 30 19:17:48.222594 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:48.222563    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:52.442230 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:52.442185    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:52.956586 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:52.956546    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:53.137732 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:53.137693    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:53.494714 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:53.494681    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"]
Oct 30 19:17:53.495287 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:53.494860    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx" podUID="1dd29180-7684-4009-9975-1be5c491d9a8" containerName="maas-api" containerID="cri-o://fb335bd6655853bd67ad0c2559d648faea6ec4b3b9c2511a24b2722db5a9538e" gracePeriod=30
Oct 30 19:17:53.630897 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:53.630864    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:53.804889 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:53.804865    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"
Oct 30 19:17:53.851260 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:53.851231    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1dd29180-7684-4009-9975-1be5c491d9a8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dd29180-7684-4009-9975-1be5c491d9a8-maas-api-tls\") pod \"1dd29180-7684-4009-9975-1be5c491d9a8\" (UID: \"1dd29180-7684-4009-9975-1be5c491d9a8\") "
Oct 30 19:17:53.851404 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:53.851290    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1dd29180-7684-4009-9975-1be5c491d9a8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dd29180-7684-4009-9975-1be5c491d9a8-metrics-certs\") pod \"1dd29180-7684-4009-9975-1be5c491d9a8\" (UID: \"1dd29180-7684-4009-9975-1be5c491d9a8\") "
Oct 30 19:17:53.851404 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:53.851321    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1dd29180-7684-4009-9975-1be5c491d9a8-kube-api-access-7h479\" (UniqueName: \"kubernetes.io/projected/1dd29180-7684-4009-9975-1be5c491d9a8-kube-api-access-7h479\") pod \"1dd29180-7684-4009-9975-1be5c491d9a8\" (UID: \"1dd29180-7684-4009-9975-1be5c491d9a8\") "
Oct 30 19:17:53.851404 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:53.851362    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1dd29180-7684-4009-9975-1be5c491d9a8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dd29180-7684-4009-9975-1be5c491d9a8-openshift-service-ca\") pod \"1dd29180-7684-4009-9975-1be5c491d9a8\" (UID: \"1dd29180-7684-4009-9975-1be5c491d9a8\") "
Oct 30 19:17:53.851826 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:53.851799    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1dd29180-7684-4009-9975-1be5c491d9a8-openshift-service-ca" pod "1dd29180-7684-4009-9975-1be5c491d9a8" (UID: "1dd29180-7684-4009-9975-1be5c491d9a8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:53.851994 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:53.851928    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dd29180-7684-4009-9975-1be5c491d9a8-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:53.853639 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:53.853616    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dd29180-7684-4009-9975-1be5c491d9a8-maas-api-tls" pod "1dd29180-7684-4009-9975-1be5c491d9a8" (UID: "1dd29180-7684-4009-9975-1be5c491d9a8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:17:53.854164 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:53.854140    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dd29180-7684-4009-9975-1be5c491d9a8-metrics-certs" pod "1dd29180-7684-4009-9975-1be5c491d9a8" (UID: "1dd29180-7684-4009-9975-1be5c491d9a8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:17:53.854164 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:53.854147    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dd29180-7684-4009-9975-1be5c491d9a8-kube-api-access-7h479" pod "1dd29180-7684-4009-9975-1be5c491d9a8" (UID: "1dd29180-7684-4009-9975-1be5c491d9a8"). InnerVolumeSpecName "kube-api-access-7h479". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:53.953100 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:53.953067    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dd29180-7684-4009-9975-1be5c491d9a8-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:53.953100 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:53.953093    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dd29180-7684-4009-9975-1be5c491d9a8-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:53.953100 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:53.953104    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7h479\" (UniqueName: \"kubernetes.io/projected/1dd29180-7684-4009-9975-1be5c491d9a8-kube-api-access-7h479\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:54.068110 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:54.068075    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:54.191149 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:54.191110    5820 generic.go:356] "Generic (PLEG): container finished" podID="1dd29180-7684-4009-9975-1be5c491d9a8" containerID="fb335bd6655853bd67ad0c2559d648faea6ec4b3b9c2511a24b2722db5a9538e" exitCode=0
Oct 30 19:17:54.191318 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:54.191161    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"
Oct 30 19:17:54.191318 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:54.191160    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx" event={"ID":"1dd29180-7684-4009-9975-1be5c491d9a8","Type":"ContainerDied","Data":"fb335bd6655853bd67ad0c2559d648faea6ec4b3b9c2511a24b2722db5a9538e"}
Oct 30 19:17:54.191318 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:54.191216    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx" event={"ID":"1dd29180-7684-4009-9975-1be5c491d9a8","Type":"ContainerDied","Data":"696c8410df0e392ec641e7952f2ff3cf1ab0dbeb207a4d702bb5b63091b8d414"}
Oct 30 19:17:54.191318 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:54.191247    5820 scope.go:122] "RemoveContainer" containerID="fb335bd6655853bd67ad0c2559d648faea6ec4b3b9c2511a24b2722db5a9538e"
Oct 30 19:17:54.201126 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:54.201063    5820 scope.go:122] "RemoveContainer" containerID="fb335bd6655853bd67ad0c2559d648faea6ec4b3b9c2511a24b2722db5a9538e"
Oct 30 19:17:54.201344 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:17:54.201325    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb335bd6655853bd67ad0c2559d648faea6ec4b3b9c2511a24b2722db5a9538e\": container with ID starting with fb335bd6655853bd67ad0c2559d648faea6ec4b3b9c2511a24b2722db5a9538e not found: ID does not exist" containerID="fb335bd6655853bd67ad0c2559d648faea6ec4b3b9c2511a24b2722db5a9538e"
Oct 30 19:17:54.201389 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:54.201353    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb335bd6655853bd67ad0c2559d648faea6ec4b3b9c2511a24b2722db5a9538e"} err="failed to get container status \"fb335bd6655853bd67ad0c2559d648faea6ec4b3b9c2511a24b2722db5a9538e\": rpc error: code = NotFound desc = could not find container \"fb335bd6655853bd67ad0c2559d648faea6ec4b3b9c2511a24b2722db5a9538e\": container with ID starting with fb335bd6655853bd67ad0c2559d648faea6ec4b3b9c2511a24b2722db5a9538e not found: ID does not exist"
Oct 30 19:17:54.355691 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:54.355604    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"]
Oct 30 19:17:54.376005 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:54.375767    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:54.404788 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:54.404754    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-6d96a6-6b7d49c95-tgwmx"]
Oct 30 19:17:55.020801 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:55.020763    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:55.345240 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:55.345203    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:55.438094 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:55.438062    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1dd29180-7684-4009-9975-1be5c491d9a8" path="/var/lib/kubelet/pods/1dd29180-7684-4009-9975-1be5c491d9a8/volumes"
Oct 30 19:17:55.691210 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:55.691106    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:55.887933 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:55.887892    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:56.398336 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:17:56.398301    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:01.184522 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:01.184481    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:01.995106 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:01.994826    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:02.226093 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:02.225996    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:02.284459 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:02.284422    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"]
Oct 30 19:18:02.284744 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:02.284717    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx" podUID="31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" containerName="istio-proxy" containerID="cri-o://ff3490f62ae959e8616aee05b2cd4cfb1d1feaeb669ec868f8d386e6cd096c6d" gracePeriod=30
Oct 30 19:18:02.440996 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:02.440939    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:02.666349 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:02.666318    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:04.047304 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:04.047212    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:04.242866 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:04.242824    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:04.392403 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:04.392362    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"]
Oct 30 19:18:04.392609 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:04.392585    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b" podUID="126f2139-f087-4162-af9d-ca9edd5c83da" containerName="istio-proxy" containerID="cri-o://99c82650cfe031bcb04f334773c9e31106b3c1b4eac2b49e387f27cb568c2ef9" gracePeriod=30
Oct 30 19:18:04.467815 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:04.467787    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:07.553091 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.553038    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:18:07.602959 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.602927    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istiod-ca-cert\") pod \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") "
Oct 30 19:18:07.603091 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.602984    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-podinfo\") pod \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") "
Oct 30 19:18:07.603091 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.603018    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-data\") pod \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") "
Oct 30 19:18:07.603091 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.603074    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-credential-socket\") pod \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") "
Oct 30 19:18:07.603198 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.603146    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-workload-socket\") pod \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") "
Oct 30 19:18:07.603234 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.603195    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-envoy\") pod \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") "
Oct 30 19:18:07.603282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.603244    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-workload-certs\") pod \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") "
Oct 30 19:18:07.603339 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.603313    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istiod-ca-cert" pod "31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" (UID: "31cb668c-773d-4a1c-8264-a8f1b6f6e9b9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:18:07.603339 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.603333    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-token\" (UniqueName: \"kubernetes.io/projected/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-token\") pod \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") "
Oct 30 19:18:07.603451 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.603337    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-data" pod "31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" (UID: "31cb668c-773d-4a1c-8264-a8f1b6f6e9b9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:07.603451 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.603368    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-kube-api-access-ngd7v\" (UniqueName: \"kubernetes.io/projected/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-kube-api-access-ngd7v\") pod \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\" (UID: \"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9\") "
Oct 30 19:18:07.603451 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.603427    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-credential-socket" pod "31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" (UID: "31cb668c-773d-4a1c-8264-a8f1b6f6e9b9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:07.603619 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.603473    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-workload-socket" pod "31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" (UID: "31cb668c-773d-4a1c-8264-a8f1b6f6e9b9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:07.603691 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.603619    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-workload-certs" pod "31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" (UID: "31cb668c-773d-4a1c-8264-a8f1b6f6e9b9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:07.603856 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.603834    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:07.604113 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.603862    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:07.604113 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.603876    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:07.604113 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.603890    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:07.604113 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.603908    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:07.605455 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.605430    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-podinfo" pod "31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" (UID: "31cb668c-773d-4a1c-8264-a8f1b6f6e9b9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:18:07.605663 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.605639    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-envoy" pod "31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" (UID: "31cb668c-773d-4a1c-8264-a8f1b6f6e9b9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:07.605748 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.605688    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-kube-api-access-ngd7v" pod "31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" (UID: "31cb668c-773d-4a1c-8264-a8f1b6f6e9b9"). InnerVolumeSpecName "kube-api-access-ngd7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:07.605748 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.605688    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-token" pod "31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" (UID: "31cb668c-773d-4a1c-8264-a8f1b6f6e9b9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:07.705322 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.705287    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:07.705322 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.705318    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngd7v\" (UniqueName: \"kubernetes.io/projected/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-kube-api-access-ngd7v\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:07.705524 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.705332    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:07.705524 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.705344    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:07.880074 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:07.879768    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:08.203530 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:18:08.203441    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:18:08.274564 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:08.274528    5820 generic.go:356] "Generic (PLEG): container finished" podID="31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" containerID="ff3490f62ae959e8616aee05b2cd4cfb1d1feaeb669ec868f8d386e6cd096c6d" exitCode=0
Oct 30 19:18:08.274741 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:08.274585    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"
Oct 30 19:18:08.274741 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:08.274605    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx" event={"ID":"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9","Type":"ContainerDied","Data":"ff3490f62ae959e8616aee05b2cd4cfb1d1feaeb669ec868f8d386e6cd096c6d"}
Oct 30 19:18:08.274741 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:08.274659    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx" event={"ID":"31cb668c-773d-4a1c-8264-a8f1b6f6e9b9","Type":"ContainerDied","Data":"a7a7a3be054e37ae3d850fb24380594c9a798b6f7857341f4420b1436175f957"}
Oct 30 19:18:08.274741 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:08.274679    5820 scope.go:122] "RemoveContainer" containerID="ff3490f62ae959e8616aee05b2cd4cfb1d1feaeb669ec868f8d386e6cd096c6d"
Oct 30 19:18:08.285525 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:08.285505    5820 scope.go:122] "RemoveContainer" containerID="443e08ecf06fa9576b471d7ac215f5aff6534ae8cc9faf58c00847e117a32811"
Oct 30 19:18:08.294734 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:08.294718    5820 scope.go:122] "RemoveContainer" containerID="ff3490f62ae959e8616aee05b2cd4cfb1d1feaeb669ec868f8d386e6cd096c6d"
Oct 30 19:18:08.295024 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:18:08.295001    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff3490f62ae959e8616aee05b2cd4cfb1d1feaeb669ec868f8d386e6cd096c6d\": container with ID starting with ff3490f62ae959e8616aee05b2cd4cfb1d1feaeb669ec868f8d386e6cd096c6d not found: ID does not exist" containerID="ff3490f62ae959e8616aee05b2cd4cfb1d1feaeb669ec868f8d386e6cd096c6d"
Oct 30 19:18:08.295086 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:08.295034    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff3490f62ae959e8616aee05b2cd4cfb1d1feaeb669ec868f8d386e6cd096c6d"} err="failed to get container status \"ff3490f62ae959e8616aee05b2cd4cfb1d1feaeb669ec868f8d386e6cd096c6d\": rpc error: code = NotFound desc = could not find container \"ff3490f62ae959e8616aee05b2cd4cfb1d1feaeb669ec868f8d386e6cd096c6d\": container with ID starting with ff3490f62ae959e8616aee05b2cd4cfb1d1feaeb669ec868f8d386e6cd096c6d not found: ID does not exist"
Oct 30 19:18:08.295086 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:08.295050    5820 scope.go:122] "RemoveContainer" containerID="443e08ecf06fa9576b471d7ac215f5aff6534ae8cc9faf58c00847e117a32811"
Oct 30 19:18:08.295271 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:18:08.295254    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"443e08ecf06fa9576b471d7ac215f5aff6534ae8cc9faf58c00847e117a32811\": container with ID starting with 443e08ecf06fa9576b471d7ac215f5aff6534ae8cc9faf58c00847e117a32811 not found: ID does not exist" containerID="443e08ecf06fa9576b471d7ac215f5aff6534ae8cc9faf58c00847e117a32811"
Oct 30 19:18:08.295316 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:08.295279    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"443e08ecf06fa9576b471d7ac215f5aff6534ae8cc9faf58c00847e117a32811"} err="failed to get container status \"443e08ecf06fa9576b471d7ac215f5aff6534ae8cc9faf58c00847e117a32811\": rpc error: code = NotFound desc = could not find container \"443e08ecf06fa9576b471d7ac215f5aff6534ae8cc9faf58c00847e117a32811\": container with ID starting with 443e08ecf06fa9576b471d7ac215f5aff6534ae8cc9faf58c00847e117a32811 not found: ID does not exist"
Oct 30 19:18:08.439780 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:08.439749    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"]
Oct 30 19:18:08.488989 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:08.488872    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-52d383-openshift-default-6dfd5748df-glbtx"]
Oct 30 19:18:09.442859 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.442823    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" path="/var/lib/kubelet/pods/31cb668c-773d-4a1c-8264-a8f1b6f6e9b9/volumes"
Oct 30 19:18:09.458229 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.458202    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"]
Oct 30 19:18:09.458809 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.458789    5820 cpu_manager.go:414] "containerMap: removing container" podUID="31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" containerName="istio-proxy"
Oct 30 19:18:09.458904 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.458810    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" containerName="istio-proxy"
Oct 30 19:18:09.458904 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.458844    5820 cpu_manager.go:414] "containerMap: removing container" podUID="31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" containerName="istio-proxy"
Oct 30 19:18:09.458904 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.458851    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" containerName="istio-proxy"
Oct 30 19:18:09.458904 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.458881    5820 cpu_manager.go:414] "containerMap: removing container" podUID="ce7148f2-3098-45df-bf88-4edfc8117df0" containerName="istio-proxy"
Oct 30 19:18:09.458904 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.458889    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce7148f2-3098-45df-bf88-4edfc8117df0" containerName="istio-proxy"
Oct 30 19:18:09.459195 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.458908    5820 cpu_manager.go:414] "containerMap: removing container" podUID="1dd29180-7684-4009-9975-1be5c491d9a8" containerName="maas-api"
Oct 30 19:18:09.459195 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.458915    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1dd29180-7684-4009-9975-1be5c491d9a8" containerName="maas-api"
Oct 30 19:18:09.459195 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.459064    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" containerName="istio-proxy"
Oct 30 19:18:09.459195 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.459085    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="31cb668c-773d-4a1c-8264-a8f1b6f6e9b9" containerName="istio-proxy"
Oct 30 19:18:09.459195 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.459101    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce7148f2-3098-45df-bf88-4edfc8117df0" containerName="istio-proxy"
Oct 30 19:18:09.459195 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.459112    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="1dd29180-7684-4009-9975-1be5c491d9a8" containerName="maas-api"
Oct 30 19:18:09.462664 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.462640    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.525067 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.525034    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"]
Oct 30 19:18:09.525691 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.525658    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.525842 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.525809    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.525917 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.525852    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.525917 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.525888    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.526038 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.526015    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25b788f8-ea78-4bb2-a154-b30f48496c04-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.526151 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.526128    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.526256 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.526185    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsjl7\" (UniqueName: \"kubernetes.io/projected/25b788f8-ea78-4bb2-a154-b30f48496c04-kube-api-access-qsjl7\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.526302 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.526270    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.526383 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.526337    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.627682 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.627648    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.627819 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.627704    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.627819 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.627791    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.628016 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.627873    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.628016 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.627903    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.628016 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.627960    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.628016 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.627990    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25b788f8-ea78-4bb2-a154-b30f48496c04-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.628252 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.628084    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.628252 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.628139    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsjl7\" (UniqueName: \"kubernetes.io/projected/25b788f8-ea78-4bb2-a154-b30f48496c04-kube-api-access-qsjl7\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.628374 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.628281    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.628986 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.628937    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25b788f8-ea78-4bb2-a154-b30f48496c04-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.629252 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.629229    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.629342 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.629283    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.629433 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.629410    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.630333 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.630307    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.630558 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.630542    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.660712 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.660626    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:18:09.692452 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.692417    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.692452 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.692449    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsjl7\" (UniqueName: \"kubernetes.io/projected/25b788f8-ea78-4bb2-a154-b30f48496c04-kube-api-access-qsjl7\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.729076 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.729045    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/126f2139-f087-4162-af9d-ca9edd5c83da-istio-token\" (UniqueName: \"kubernetes.io/projected/126f2139-f087-4162-af9d-ca9edd5c83da-istio-token\") pod \"126f2139-f087-4162-af9d-ca9edd5c83da\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") "
Oct 30 19:18:09.729230 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.729099    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-istio-data\") pod \"126f2139-f087-4162-af9d-ca9edd5c83da\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") "
Oct 30 19:18:09.729230 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.729179    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-istio-envoy\") pod \"126f2139-f087-4162-af9d-ca9edd5c83da\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") "
Oct 30 19:18:09.729230 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.729214    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/126f2139-f087-4162-af9d-ca9edd5c83da-kube-api-access-bj4sv\" (UniqueName: \"kubernetes.io/projected/126f2139-f087-4162-af9d-ca9edd5c83da-kube-api-access-bj4sv\") pod \"126f2139-f087-4162-af9d-ca9edd5c83da\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") "
Oct 30 19:18:09.729405 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.729290    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-workload-socket\") pod \"126f2139-f087-4162-af9d-ca9edd5c83da\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") "
Oct 30 19:18:09.729405 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.729348    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-workload-certs\") pod \"126f2139-f087-4162-af9d-ca9edd5c83da\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") "
Oct 30 19:18:09.729509 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.729414    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-credential-socket\") pod \"126f2139-f087-4162-af9d-ca9edd5c83da\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") "
Oct 30 19:18:09.729509 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.729436    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-istio-data" pod "126f2139-f087-4162-af9d-ca9edd5c83da" (UID: "126f2139-f087-4162-af9d-ca9edd5c83da"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:09.729509 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.729479    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/126f2139-f087-4162-af9d-ca9edd5c83da-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/126f2139-f087-4162-af9d-ca9edd5c83da-istiod-ca-cert\") pod \"126f2139-f087-4162-af9d-ca9edd5c83da\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") "
Oct 30 19:18:09.729662 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.729528    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/126f2139-f087-4162-af9d-ca9edd5c83da-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/126f2139-f087-4162-af9d-ca9edd5c83da-istio-podinfo\") pod \"126f2139-f087-4162-af9d-ca9edd5c83da\" (UID: \"126f2139-f087-4162-af9d-ca9edd5c83da\") "
Oct 30 19:18:09.729662 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.729612    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-workload-certs" pod "126f2139-f087-4162-af9d-ca9edd5c83da" (UID: "126f2139-f087-4162-af9d-ca9edd5c83da"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:09.729662 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.729642    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-workload-socket" pod "126f2139-f087-4162-af9d-ca9edd5c83da" (UID: "126f2139-f087-4162-af9d-ca9edd5c83da"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:09.729799 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.729720    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-credential-socket" pod "126f2139-f087-4162-af9d-ca9edd5c83da" (UID: "126f2139-f087-4162-af9d-ca9edd5c83da"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:09.729930 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.729911    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/126f2139-f087-4162-af9d-ca9edd5c83da-istiod-ca-cert" pod "126f2139-f087-4162-af9d-ca9edd5c83da" (UID: "126f2139-f087-4162-af9d-ca9edd5c83da"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:18:09.730057 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.730044    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:09.730101 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.730060    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:09.730101 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.730068    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:09.730101 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.730077    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:09.730101 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.730084    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/126f2139-f087-4162-af9d-ca9edd5c83da-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:09.731434 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.731410    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/126f2139-f087-4162-af9d-ca9edd5c83da-istio-token" pod "126f2139-f087-4162-af9d-ca9edd5c83da" (UID: "126f2139-f087-4162-af9d-ca9edd5c83da"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:09.731998 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.731971    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/126f2139-f087-4162-af9d-ca9edd5c83da-istio-podinfo" pod "126f2139-f087-4162-af9d-ca9edd5c83da" (UID: "126f2139-f087-4162-af9d-ca9edd5c83da"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:18:09.731998 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.731980    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-istio-envoy" pod "126f2139-f087-4162-af9d-ca9edd5c83da" (UID: "126f2139-f087-4162-af9d-ca9edd5c83da"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:09.731998 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.731995    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/126f2139-f087-4162-af9d-ca9edd5c83da-kube-api-access-bj4sv" pod "126f2139-f087-4162-af9d-ca9edd5c83da" (UID: "126f2139-f087-4162-af9d-ca9edd5c83da"). InnerVolumeSpecName "kube-api-access-bj4sv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:09.790443 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.790411    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:09.831060 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.831028    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/126f2139-f087-4162-af9d-ca9edd5c83da-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:09.831060 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.831056    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/126f2139-f087-4162-af9d-ca9edd5c83da-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:09.831060 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.831065    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/126f2139-f087-4162-af9d-ca9edd5c83da-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:09.831289 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:09.831074    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bj4sv\" (UniqueName: \"kubernetes.io/projected/126f2139-f087-4162-af9d-ca9edd5c83da-kube-api-access-bj4sv\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:10.221200 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:10.221172    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"]
Oct 30 19:18:10.223457 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:18:10.223430    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25b788f8_ea78_4bb2_a154_b30f48496c04.slice/crio-6670c3aa01a922e1c7ca7f1a8522d1e62f2a1726d29a0d8730f54452538e5dd1 WatchSource:0}: Error finding container 6670c3aa01a922e1c7ca7f1a8522d1e62f2a1726d29a0d8730f54452538e5dd1: Status 404 returned error can't find the container with id 6670c3aa01a922e1c7ca7f1a8522d1e62f2a1726d29a0d8730f54452538e5dd1
Oct 30 19:18:10.225849 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:10.225819    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:18:10.225933 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:10.225881    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:18:10.225933 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:10.225911    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:18:10.298711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:10.298678    5820 generic.go:356] "Generic (PLEG): container finished" podID="126f2139-f087-4162-af9d-ca9edd5c83da" containerID="99c82650cfe031bcb04f334773c9e31106b3c1b4eac2b49e387f27cb568c2ef9" exitCode=0
Oct 30 19:18:10.298864 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:10.298738    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"
Oct 30 19:18:10.298864 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:10.298746    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b" event={"ID":"126f2139-f087-4162-af9d-ca9edd5c83da","Type":"ContainerDied","Data":"99c82650cfe031bcb04f334773c9e31106b3c1b4eac2b49e387f27cb568c2ef9"}
Oct 30 19:18:10.298864 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:10.298834    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b" event={"ID":"126f2139-f087-4162-af9d-ca9edd5c83da","Type":"ContainerDied","Data":"6769174f8c703a056130a8d0740253e8b56ad60fee1a5767283d8d9d3b4bd1d1"}
Oct 30 19:18:10.299080 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:10.298865    5820 scope.go:122] "RemoveContainer" containerID="99c82650cfe031bcb04f334773c9e31106b3c1b4eac2b49e387f27cb568c2ef9"
Oct 30 19:18:10.302308 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:10.302271    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4" event={"ID":"25b788f8-ea78-4bb2-a154-b30f48496c04","Type":"ContainerStarted","Data":"10bb75991d3d362be456807ea12c292e6cbc78ba8336bf8ae77c87d86a263c0a"}
Oct 30 19:18:10.302429 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:10.302356    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4" event={"ID":"25b788f8-ea78-4bb2-a154-b30f48496c04","Type":"ContainerStarted","Data":"6670c3aa01a922e1c7ca7f1a8522d1e62f2a1726d29a0d8730f54452538e5dd1"}
Oct 30 19:18:10.309565 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:10.309546    5820 scope.go:122] "RemoveContainer" containerID="2531213a74bf757ed94e1886fbd2cf3e7bc04993c5c4a70823a4401f767f3e6c"
Oct 30 19:18:10.320171 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:10.320141    5820 scope.go:122] "RemoveContainer" containerID="99c82650cfe031bcb04f334773c9e31106b3c1b4eac2b49e387f27cb568c2ef9"
Oct 30 19:18:10.320446 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:18:10.320423    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99c82650cfe031bcb04f334773c9e31106b3c1b4eac2b49e387f27cb568c2ef9\": container with ID starting with 99c82650cfe031bcb04f334773c9e31106b3c1b4eac2b49e387f27cb568c2ef9 not found: ID does not exist" containerID="99c82650cfe031bcb04f334773c9e31106b3c1b4eac2b49e387f27cb568c2ef9"
Oct 30 19:18:10.320513 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:10.320455    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99c82650cfe031bcb04f334773c9e31106b3c1b4eac2b49e387f27cb568c2ef9"} err="failed to get container status \"99c82650cfe031bcb04f334773c9e31106b3c1b4eac2b49e387f27cb568c2ef9\": rpc error: code = NotFound desc = could not find container \"99c82650cfe031bcb04f334773c9e31106b3c1b4eac2b49e387f27cb568c2ef9\": container with ID starting with 99c82650cfe031bcb04f334773c9e31106b3c1b4eac2b49e387f27cb568c2ef9 not found: ID does not exist"
Oct 30 19:18:10.320513 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:10.320472    5820 scope.go:122] "RemoveContainer" containerID="2531213a74bf757ed94e1886fbd2cf3e7bc04993c5c4a70823a4401f767f3e6c"
Oct 30 19:18:10.320695 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:18:10.320681    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2531213a74bf757ed94e1886fbd2cf3e7bc04993c5c4a70823a4401f767f3e6c\": container with ID starting with 2531213a74bf757ed94e1886fbd2cf3e7bc04993c5c4a70823a4401f767f3e6c not found: ID does not exist" containerID="2531213a74bf757ed94e1886fbd2cf3e7bc04993c5c4a70823a4401f767f3e6c"
Oct 30 19:18:10.320748 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:10.320698    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2531213a74bf757ed94e1886fbd2cf3e7bc04993c5c4a70823a4401f767f3e6c"} err="failed to get container status \"2531213a74bf757ed94e1886fbd2cf3e7bc04993c5c4a70823a4401f767f3e6c\": rpc error: code = NotFound desc = could not find container \"2531213a74bf757ed94e1886fbd2cf3e7bc04993c5c4a70823a4401f767f3e6c\": container with ID starting with 2531213a74bf757ed94e1886fbd2cf3e7bc04993c5c4a70823a4401f767f3e6c not found: ID does not exist"
Oct 30 19:18:10.465385 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:10.465347    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"]
Oct 30 19:18:10.469843 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:10.469799    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-6d96a6-openshift-default-bdf96d4d5-r5r5b"]
Oct 30 19:18:10.589133 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:10.589080    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4" podStartSLOduration=1.5890659390000001 podStartE2EDuration="1.589065939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:10.586526982 +0000 UTC m=+2611.298275678" watchObservedRunningTime="2026-10-30 19:18:10.589065939 +0000 UTC m=+2611.300814699"
Oct 30 19:18:10.791266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:10.791239    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:11.456976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:11.437839    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="126f2139-f087-4162-af9d-ca9edd5c83da" path="/var/lib/kubelet/pods/126f2139-f087-4162-af9d-ca9edd5c83da/volumes"
Oct 30 19:18:11.792945 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:11.792857    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:18:11.792945 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:11.792912    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4" podUID="25b788f8-ea78-4bb2-a154-b30f48496c04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused"
Oct 30 19:18:12.460347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.460062    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"]
Oct 30 19:18:12.460987 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.460962    5820 cpu_manager.go:414] "containerMap: removing container" podUID="126f2139-f087-4162-af9d-ca9edd5c83da" containerName="istio-proxy"
Oct 30 19:18:12.460987 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.460986    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="126f2139-f087-4162-af9d-ca9edd5c83da" containerName="istio-proxy"
Oct 30 19:18:12.461213 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.461018    5820 cpu_manager.go:414] "containerMap: removing container" podUID="126f2139-f087-4162-af9d-ca9edd5c83da" containerName="istio-proxy"
Oct 30 19:18:12.461213 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.461027    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="126f2139-f087-4162-af9d-ca9edd5c83da" containerName="istio-proxy"
Oct 30 19:18:12.461213 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.461154    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="126f2139-f087-4162-af9d-ca9edd5c83da" containerName="istio-proxy"
Oct 30 19:18:12.461213 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.461178    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="126f2139-f087-4162-af9d-ca9edd5c83da" containerName="istio-proxy"
Oct 30 19:18:12.466500 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.466481    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"
Oct 30 19:18:12.505007 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.504980    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"]
Oct 30 19:18:12.518414 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.518393    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-6d953c-serving-cert\""
Oct 30 19:18:12.577364 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.577332    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3393cba7-4779-4143-b56e-879607b91b1b-metrics-certs\") pod \"maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97\" (UID: \"3393cba7-4779-4143-b56e-879607b91b1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"
Oct 30 19:18:12.577696 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.577665    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3393cba7-4779-4143-b56e-879607b91b1b-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97\" (UID: \"3393cba7-4779-4143-b56e-879607b91b1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"
Oct 30 19:18:12.577860 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.577837    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h82jd\" (UniqueName: \"kubernetes.io/projected/3393cba7-4779-4143-b56e-879607b91b1b-kube-api-access-h82jd\") pod \"maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97\" (UID: \"3393cba7-4779-4143-b56e-879607b91b1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"
Oct 30 19:18:12.577999 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.577975    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3393cba7-4779-4143-b56e-879607b91b1b-maas-api-tls\") pod \"maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97\" (UID: \"3393cba7-4779-4143-b56e-879607b91b1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"
Oct 30 19:18:12.679017 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.678976    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3393cba7-4779-4143-b56e-879607b91b1b-metrics-certs\") pod \"maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97\" (UID: \"3393cba7-4779-4143-b56e-879607b91b1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"
Oct 30 19:18:12.679192 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.679096    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3393cba7-4779-4143-b56e-879607b91b1b-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97\" (UID: \"3393cba7-4779-4143-b56e-879607b91b1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"
Oct 30 19:18:12.679192 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.679145    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h82jd\" (UniqueName: \"kubernetes.io/projected/3393cba7-4779-4143-b56e-879607b91b1b-kube-api-access-h82jd\") pod \"maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97\" (UID: \"3393cba7-4779-4143-b56e-879607b91b1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"
Oct 30 19:18:12.679192 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.679176    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3393cba7-4779-4143-b56e-879607b91b1b-maas-api-tls\") pod \"maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97\" (UID: \"3393cba7-4779-4143-b56e-879607b91b1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"
Oct 30 19:18:12.679395 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:18:12.679373    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-serving-cert: secret "maas-api-e2e-praxis-a-6d953c-serving-cert" not found
Oct 30 19:18:12.679454 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:18:12.679441    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3393cba7-4779-4143-b56e-879607b91b1b-maas-api-tls podName:3393cba7-4779-4143-b56e-879607b91b1b nodeName:}" failed. No retries permitted until 2026-10-30 19:18:13.179419696 +0000 UTC m=+2613.891168387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3393cba7-4779-4143-b56e-879607b91b1b-maas-api-tls") pod "maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97" (UID: "3393cba7-4779-4143-b56e-879607b91b1b") : secret "maas-api-e2e-praxis-a-6d953c-serving-cert" not found
Oct 30 19:18:12.679795 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.679770    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3393cba7-4779-4143-b56e-879607b91b1b-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97\" (UID: \"3393cba7-4779-4143-b56e-879607b91b1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"
Oct 30 19:18:12.681815 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.681783    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3393cba7-4779-4143-b56e-879607b91b1b-metrics-certs\") pod \"maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97\" (UID: \"3393cba7-4779-4143-b56e-879607b91b1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"
Oct 30 19:18:12.746578 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.745282    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:12.751057 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.750995    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h82jd\" (UniqueName: \"kubernetes.io/projected/3393cba7-4779-4143-b56e-879607b91b1b-kube-api-access-h82jd\") pod \"maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97\" (UID: \"3393cba7-4779-4143-b56e-879607b91b1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"
Oct 30 19:18:12.791022 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.790990    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:18:12.791219 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.791053    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4" podUID="25b788f8-ea78-4bb2-a154-b30f48496c04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused"
Oct 30 19:18:12.995535 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:12.995494    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:13.187077 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:13.187041    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3393cba7-4779-4143-b56e-879607b91b1b-maas-api-tls\") pod \"maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97\" (UID: \"3393cba7-4779-4143-b56e-879607b91b1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"
Oct 30 19:18:13.189630 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:13.189611    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3393cba7-4779-4143-b56e-879607b91b1b-maas-api-tls\") pod \"maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97\" (UID: \"3393cba7-4779-4143-b56e-879607b91b1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"
Oct 30 19:18:13.380491 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:13.380451    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"
Oct 30 19:18:13.782282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:13.782236    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"]
Oct 30 19:18:13.787988 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:13.787921    5820 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:18:13.791235 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:13.791211    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:18:13.791359 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:13.791253    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4" podUID="25b788f8-ea78-4bb2-a154-b30f48496c04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused"
Oct 30 19:18:14.327328 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:14.327296    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97" event={"ID":"3393cba7-4779-4143-b56e-879607b91b1b","Type":"ContainerStarted","Data":"fb91df8b6c45d43c36a972c737647a4137a127ecdd4dc703a5cd9cd7c3d6697c"}
Oct 30 19:18:14.790906 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:14.790877    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:18:14.791082 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:14.790928    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4" podUID="25b788f8-ea78-4bb2-a154-b30f48496c04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused"
Oct 30 19:18:15.359603 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:15.359558    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97" event={"ID":"3393cba7-4779-4143-b56e-879607b91b1b","Type":"ContainerStarted","Data":"29ee946faa835ac6865a72d4097f1cffca99d9b0335c5af792eb061769cda34a"}
Oct 30 19:18:15.360077 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:15.359728    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"
Oct 30 19:18:15.791662 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:15.791583    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:18:15.791662 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:15.791647    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4" podUID="25b788f8-ea78-4bb2-a154-b30f48496c04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused"
Oct 30 19:18:16.791290 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:16.791253    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:18:16.791793 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:16.791310    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4" podUID="25b788f8-ea78-4bb2-a154-b30f48496c04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused"
Oct 30 19:18:17.649058 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:17.648284    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:17.795120 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:17.795088    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:17.795518 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:17.795374    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:17.795911 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:17.795894    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:18:17.965554 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:17.965456    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97" podStartSLOduration=5.051234946 podStartE2EDuration="5.96544364s" totalImagesPullingTime="914.208694ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:15.498576182 +0000 UTC m=+2616.210324879" watchObservedRunningTime="2026-10-30 19:18:17.96544364 +0000 UTC m=+2618.677192336"
Oct 30 19:18:21.371513 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:21.371481    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"
Oct 30 19:18:22.538980 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:22.538888    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:22.783372 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:22.783338    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:22.975455 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:22.975417    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:27.344090 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:27.344051    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:32.473555 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:32.473521    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:36.535642 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:36.535608    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"]
Oct 30 19:18:36.536137 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:36.535837    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg" podUID="0bb5a3ba-012c-41e0-bc65-e12c59bc9e15" containerName="maas-api" containerID="cri-o://69e770d8e6b2c4e27e27bb86306cfb62f503a404b49b7d10df129d8db1c96613" gracePeriod=30
Oct 30 19:18:36.672964 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:36.672912    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:36.825061 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:36.825036    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"
Oct 30 19:18:36.829102 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:36.829082    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-openshift-service-ca\") pod \"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15\" (UID: \"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15\") "
Oct 30 19:18:36.829218 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:36.829117    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-kube-api-access-75wbm\" (UniqueName: \"kubernetes.io/projected/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-kube-api-access-75wbm\") pod \"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15\" (UID: \"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15\") "
Oct 30 19:18:36.829282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:36.829215    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-maas-api-tls\") pod \"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15\" (UID: \"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15\") "
Oct 30 19:18:36.829340 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:36.829288    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-metrics-certs\") pod \"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15\" (UID: \"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15\") "
Oct 30 19:18:36.829419 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:36.829394    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-openshift-service-ca" pod "0bb5a3ba-012c-41e0-bc65-e12c59bc9e15" (UID: "0bb5a3ba-012c-41e0-bc65-e12c59bc9e15"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:18:36.829735 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:36.829719    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:36.831366 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:36.831345    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-maas-api-tls" pod "0bb5a3ba-012c-41e0-bc65-e12c59bc9e15" (UID: "0bb5a3ba-012c-41e0-bc65-e12c59bc9e15"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:18:36.831446 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:36.831381    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-metrics-certs" pod "0bb5a3ba-012c-41e0-bc65-e12c59bc9e15" (UID: "0bb5a3ba-012c-41e0-bc65-e12c59bc9e15"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:18:36.831514 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:36.831496    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-kube-api-access-75wbm" pod "0bb5a3ba-012c-41e0-bc65-e12c59bc9e15" (UID: "0bb5a3ba-012c-41e0-bc65-e12c59bc9e15"). InnerVolumeSpecName "kube-api-access-75wbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:36.936491 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:36.936147    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:36.936491 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:36.936187    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:36.936491 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:36.936203    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75wbm\" (UniqueName: \"kubernetes.io/projected/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15-kube-api-access-75wbm\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:37.192653 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:37.192570    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:37.368297 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:37.368258    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:37.538877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:37.538843    5820 generic.go:356] "Generic (PLEG): container finished" podID="0bb5a3ba-012c-41e0-bc65-e12c59bc9e15" containerID="69e770d8e6b2c4e27e27bb86306cfb62f503a404b49b7d10df129d8db1c96613" exitCode=0
Oct 30 19:18:37.545406 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:37.540479    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg" event={"ID":"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15","Type":"ContainerDied","Data":"69e770d8e6b2c4e27e27bb86306cfb62f503a404b49b7d10df129d8db1c96613"}
Oct 30 19:18:37.545406 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:37.541252    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg" event={"ID":"0bb5a3ba-012c-41e0-bc65-e12c59bc9e15","Type":"ContainerDied","Data":"6df583f2ce6f8e58f6b1320468c58136a1b424d21f79d03b63c686d6c6cdb68b"}
Oct 30 19:18:37.545406 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:37.541285    5820 scope.go:122] "RemoveContainer" containerID="69e770d8e6b2c4e27e27bb86306cfb62f503a404b49b7d10df129d8db1c96613"
Oct 30 19:18:37.545406 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:37.540568    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"
Oct 30 19:18:37.566436 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:37.566409    5820 scope.go:122] "RemoveContainer" containerID="69e770d8e6b2c4e27e27bb86306cfb62f503a404b49b7d10df129d8db1c96613"
Oct 30 19:18:37.566854 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:18:37.566827    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69e770d8e6b2c4e27e27bb86306cfb62f503a404b49b7d10df129d8db1c96613\": container with ID starting with 69e770d8e6b2c4e27e27bb86306cfb62f503a404b49b7d10df129d8db1c96613 not found: ID does not exist" containerID="69e770d8e6b2c4e27e27bb86306cfb62f503a404b49b7d10df129d8db1c96613"
Oct 30 19:18:37.567072 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:37.567041    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69e770d8e6b2c4e27e27bb86306cfb62f503a404b49b7d10df129d8db1c96613"} err="failed to get container status \"69e770d8e6b2c4e27e27bb86306cfb62f503a404b49b7d10df129d8db1c96613\": rpc error: code = NotFound desc = could not find container \"69e770d8e6b2c4e27e27bb86306cfb62f503a404b49b7d10df129d8db1c96613\": container with ID starting with 69e770d8e6b2c4e27e27bb86306cfb62f503a404b49b7d10df129d8db1c96613 not found: ID does not exist"
Oct 30 19:18:37.709893 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:37.709820    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"]
Oct 30 19:18:37.758433 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:37.758400    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-91cb32-d6b4f447c-gdqbg"]
Oct 30 19:18:38.204300 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:18:38.204262    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:18:38.246861 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:38.246824    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:38.512122 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:38.512038    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:38.693495 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:38.693451    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:39.204508 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:39.204468    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:39.437521 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:39.437488    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0bb5a3ba-012c-41e0-bc65-e12c59bc9e15" path="/var/lib/kubelet/pods/0bb5a3ba-012c-41e0-bc65-e12c59bc9e15/volumes"
Oct 30 19:18:43.695566 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:43.695528    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:45.390282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:45.390239    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:45.609791 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:45.609750    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:45.816723 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:45.816686    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:45.908032 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:45.907999    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"]
Oct 30 19:18:45.908592 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:45.908577    5820 cpu_manager.go:414] "containerMap: removing container" podUID="0bb5a3ba-012c-41e0-bc65-e12c59bc9e15" containerName="maas-api"
Oct 30 19:18:45.908639 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:45.908593    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0bb5a3ba-012c-41e0-bc65-e12c59bc9e15" containerName="maas-api"
Oct 30 19:18:45.908681 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:45.908670    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="0bb5a3ba-012c-41e0-bc65-e12c59bc9e15" containerName="maas-api"
Oct 30 19:18:45.913635 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:45.913616    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.024442 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.024407    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-23302c-openshift-default-dockercfg-r5rk4\""
Oct 30 19:18:46.025230 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.025200    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"]
Oct 30 19:18:46.029864 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.029839    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"]
Oct 30 19:18:46.030155 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.030078    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf" podUID="17ba2a01-0007-4f37-b1a9-464910fd82e6" containerName="istio-proxy" containerID="cri-o://a557ecfed49b1f9da927736ecd1eb28440a36b8c36e15ed5cd3573e3e2b69b9a" gracePeriod=30
Oct 30 19:18:46.045551 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.045515    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4415a3e-5238-48e7-a6b2-e97c1904381f-istiod-ca-cert\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.045697 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.045577    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-data\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.045697 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.045671    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-token\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.045805 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.045711    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-credential-socket\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.045842 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.045796    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhsx7\" (UniqueName: \"kubernetes.io/projected/f4415a3e-5238-48e7-a6b2-e97c1904381f-kube-api-access-dhsx7\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.045964 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.045931    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-workload-certs\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.046022 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.046006    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-podinfo\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.046127 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.046109    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-envoy\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.046178 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.046143    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-workload-socket\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.054746 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.054724    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:46.148474 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.147818    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-podinfo\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.148474 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.147894    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-envoy\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.148474 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.147934    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-workload-socket\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.148474 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.148005    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4415a3e-5238-48e7-a6b2-e97c1904381f-istiod-ca-cert\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.148474 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.148041    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-data\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.148474 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.148084    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-token\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.148474 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.148114    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-credential-socket\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.148474 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.148162    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhsx7\" (UniqueName: \"kubernetes.io/projected/f4415a3e-5238-48e7-a6b2-e97c1904381f-kube-api-access-dhsx7\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.148474 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.148294    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-workload-certs\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.149156 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.148824    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-data\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.149280 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.148715    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-workload-certs\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.149280 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.149258    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-credential-socket\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.149514 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.149490    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-workload-socket\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.149940 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.149911    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4415a3e-5238-48e7-a6b2-e97c1904381f-istiod-ca-cert\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.151892 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.151866    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-envoy\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.160708 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.160673    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-podinfo\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.212964 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.212923    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-token\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.217030 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.217002    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhsx7\" (UniqueName: \"kubernetes.io/projected/f4415a3e-5238-48e7-a6b2-e97c1904381f-kube-api-access-dhsx7\") pod \"e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") " pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.253434 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.253405    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:46.262959 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.262920    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"]
Oct 30 19:18:46.267233 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.267214    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.289217 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.288195    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:46.319850 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.319817    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-8f9c6b-openshift-default-dockercfg-48sks\""
Oct 30 19:18:46.338579 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.338398    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"]
Oct 30 19:18:46.350848 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.350812    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/795704e1-5f91-4636-a0b5-d9271dc4551d-istiod-ca-cert\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.351024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.350977    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-envoy\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.351099 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.351021    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-data\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.351099 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.351078    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-token\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.351899 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.351143    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-workload-socket\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.351899 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.351178    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-credential-socket\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.351899 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.351265    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-podinfo\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.351899 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.351370    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-workload-certs\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.351899 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.351477    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5khrg\" (UniqueName: \"kubernetes.io/projected/795704e1-5f91-4636-a0b5-d9271dc4551d-kube-api-access-5khrg\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.452280 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.452204    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-podinfo\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.452754 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.452281    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-workload-certs\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.452754 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.452364    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5khrg\" (UniqueName: \"kubernetes.io/projected/795704e1-5f91-4636-a0b5-d9271dc4551d-kube-api-access-5khrg\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.452754 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.452475    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/795704e1-5f91-4636-a0b5-d9271dc4551d-istiod-ca-cert\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.452754 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.452543    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-envoy\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.452754 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.452579    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-data\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.452754 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.452612    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-token\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.452754 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.452681    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-workload-socket\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.452754 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.452738    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-credential-socket\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.453275 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.452753    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-workload-certs\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.453275 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.452994    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-data\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.453275 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.453162    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-credential-socket\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.453275 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.453200    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-workload-socket\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.453412 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.453308    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/795704e1-5f91-4636-a0b5-d9271dc4551d-istiod-ca-cert\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.455243 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.455221    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-podinfo\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.455386 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.455368    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-envoy\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.513424 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.513399    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-token\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.513515 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.513478    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5khrg\" (UniqueName: \"kubernetes.io/projected/795704e1-5f91-4636-a0b5-d9271dc4551d-kube-api-access-5khrg\") pod \"e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") " pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.521161 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.521131    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:46.594914 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.594888    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:46.693486 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.693447    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"]
Oct 30 19:18:46.696438 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:18:46.696396    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4415a3e_5238_48e7_a6b2_e97c1904381f.slice/crio-d55495bccff50b7d5a0f136c4b892345d85e5eb3d7be39f2a903851cf346408a WatchSource:0}: Error finding container d55495bccff50b7d5a0f136c4b892345d85e5eb3d7be39f2a903851cf346408a: Status 404 returned error can't find the container with id d55495bccff50b7d5a0f136c4b892345d85e5eb3d7be39f2a903851cf346408a
Oct 30 19:18:46.699297 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.699256    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:18:46.699387 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.699351    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:18:46.699427 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.699402    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:18:46.733045 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:46.733021    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:47.000645 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:47.000617    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"]
Oct 30 19:18:47.002856 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:18:47.002816    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod795704e1_5f91_4636_a0b5_d9271dc4551d.slice/crio-f4ec8f4e30728313015242448217b089ec74e528c74fa26bb1107c3ba11d7d4a WatchSource:0}: Error finding container f4ec8f4e30728313015242448217b089ec74e528c74fa26bb1107c3ba11d7d4a: Status 404 returned error can't find the container with id f4ec8f4e30728313015242448217b089ec74e528c74fa26bb1107c3ba11d7d4a
Oct 30 19:18:47.005431 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:47.005401    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:18:47.005513 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:47.005465    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:18:47.005513 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:47.005493    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:18:47.661045 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:47.661006    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2" event={"ID":"795704e1-5f91-4636-a0b5-d9271dc4551d","Type":"ContainerStarted","Data":"02a2552ec01a773a0fedd7dc564aedc21963c9e9e654e0a47860038b1e93382e"}
Oct 30 19:18:47.661505 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:47.661060    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2" event={"ID":"795704e1-5f91-4636-a0b5-d9271dc4551d","Type":"ContainerStarted","Data":"f4ec8f4e30728313015242448217b089ec74e528c74fa26bb1107c3ba11d7d4a"}
Oct 30 19:18:47.662907 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:47.662879    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt" event={"ID":"f4415a3e-5238-48e7-a6b2-e97c1904381f","Type":"ContainerStarted","Data":"d0721ee98511840ec85e512dbce762684b8399067cea19a5a2054635e6c3eb53"}
Oct 30 19:18:47.663075 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:47.662923    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt" event={"ID":"f4415a3e-5238-48e7-a6b2-e97c1904381f","Type":"ContainerStarted","Data":"d55495bccff50b7d5a0f136c4b892345d85e5eb3d7be39f2a903851cf346408a"}
Oct 30 19:18:47.711897 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:47.711866    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:47.787854 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:47.787809    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2" podStartSLOduration=1.7877961500000001 podStartE2EDuration="1.78779615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:47.785217436 +0000 UTC m=+2648.496966135" watchObservedRunningTime="2026-10-30 19:18:47.78779615 +0000 UTC m=+2648.499544896"
Oct 30 19:18:48.254140 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:48.254111    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:48.259230 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:48.259208    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:48.386760 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:48.386703    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt" podStartSLOduration=3.386686664 podStartE2EDuration="3.386686664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:47.927256073 +0000 UTC m=+2648.639004783" watchObservedRunningTime="2026-10-30 19:18:48.386686664 +0000 UTC m=+2649.098435371"
Oct 30 19:18:48.578532 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:48.578487    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:48.595449 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:48.595419    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:48.601155 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:48.601132    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:48.668518 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:48.668488    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:48.668978 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:48.668543    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:48.669513 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:48.669493    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:18:48.669619 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:48.669581    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:18:49.307510 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:49.307464    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:49.578633 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:49.578550    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:50.686377 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:50.686225    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:50.889936 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:50.889898    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:51.090721 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.090691    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:51.276699 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.276668    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:51.292460 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.292439    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:18:51.433605 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.433575    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-credential-socket\") pod \"17ba2a01-0007-4f37-b1a9-464910fd82e6\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") "
Oct 30 19:18:51.433605 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.433611    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-envoy\") pod \"17ba2a01-0007-4f37-b1a9-464910fd82e6\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") "
Oct 30 19:18:51.433870 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.433642    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-workload-certs\") pod \"17ba2a01-0007-4f37-b1a9-464910fd82e6\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") "
Oct 30 19:18:51.433870 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.433691    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-workload-socket\") pod \"17ba2a01-0007-4f37-b1a9-464910fd82e6\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") "
Oct 30 19:18:51.433870 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.433741    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17ba2a01-0007-4f37-b1a9-464910fd82e6-kube-api-access-2gznz\" (UniqueName: \"kubernetes.io/projected/17ba2a01-0007-4f37-b1a9-464910fd82e6-kube-api-access-2gznz\") pod \"17ba2a01-0007-4f37-b1a9-464910fd82e6\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") "
Oct 30 19:18:51.433870 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.433803    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/17ba2a01-0007-4f37-b1a9-464910fd82e6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17ba2a01-0007-4f37-b1a9-464910fd82e6-istiod-ca-cert\") pod \"17ba2a01-0007-4f37-b1a9-464910fd82e6\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") "
Oct 30 19:18:51.433870 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.433833    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-token\" (UniqueName: \"kubernetes.io/projected/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-token\") pod \"17ba2a01-0007-4f37-b1a9-464910fd82e6\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") "
Oct 30 19:18:51.434181 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.433881    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-podinfo\") pod \"17ba2a01-0007-4f37-b1a9-464910fd82e6\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") "
Oct 30 19:18:51.434181 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.433938    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-data\") pod \"17ba2a01-0007-4f37-b1a9-464910fd82e6\" (UID: \"17ba2a01-0007-4f37-b1a9-464910fd82e6\") "
Oct 30 19:18:51.434181 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.433944    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-credential-socket" pod "17ba2a01-0007-4f37-b1a9-464910fd82e6" (UID: "17ba2a01-0007-4f37-b1a9-464910fd82e6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:51.434181 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.433993    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-workload-certs" pod "17ba2a01-0007-4f37-b1a9-464910fd82e6" (UID: "17ba2a01-0007-4f37-b1a9-464910fd82e6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:51.434181 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.434081    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-workload-socket" pod "17ba2a01-0007-4f37-b1a9-464910fd82e6" (UID: "17ba2a01-0007-4f37-b1a9-464910fd82e6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:51.434455 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.434214    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-data" pod "17ba2a01-0007-4f37-b1a9-464910fd82e6" (UID: "17ba2a01-0007-4f37-b1a9-464910fd82e6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:51.434513 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.434501    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:51.434570 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.434522    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:51.434570 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.434538    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:51.434679 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.434615    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:51.434735 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.434679    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17ba2a01-0007-4f37-b1a9-464910fd82e6-istiod-ca-cert" pod "17ba2a01-0007-4f37-b1a9-464910fd82e6" (UID: "17ba2a01-0007-4f37-b1a9-464910fd82e6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:18:51.436453 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.436430    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17ba2a01-0007-4f37-b1a9-464910fd82e6-kube-api-access-2gznz" pod "17ba2a01-0007-4f37-b1a9-464910fd82e6" (UID: "17ba2a01-0007-4f37-b1a9-464910fd82e6"). InnerVolumeSpecName "kube-api-access-2gznz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:51.436581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.436451    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-token" pod "17ba2a01-0007-4f37-b1a9-464910fd82e6" (UID: "17ba2a01-0007-4f37-b1a9-464910fd82e6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:51.436731 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.436707    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-podinfo" pod "17ba2a01-0007-4f37-b1a9-464910fd82e6" (UID: "17ba2a01-0007-4f37-b1a9-464910fd82e6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:18:51.436731 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.436711    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-envoy" pod "17ba2a01-0007-4f37-b1a9-464910fd82e6" (UID: "17ba2a01-0007-4f37-b1a9-464910fd82e6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:51.496554 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.496527    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:51.536112 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.536085    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17ba2a01-0007-4f37-b1a9-464910fd82e6-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:51.536112 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.536111    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:51.536276 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.536120    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:51.536276 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.536134    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17ba2a01-0007-4f37-b1a9-464910fd82e6-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:51.536276 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.536145    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gznz\" (UniqueName: \"kubernetes.io/projected/17ba2a01-0007-4f37-b1a9-464910fd82e6-kube-api-access-2gznz\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:18:51.688327 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.688239    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:51.698644 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.698617    5820 generic.go:356] "Generic (PLEG): container finished" podID="17ba2a01-0007-4f37-b1a9-464910fd82e6" containerID="a557ecfed49b1f9da927736ecd1eb28440a36b8c36e15ed5cd3573e3e2b69b9a" exitCode=0
Oct 30 19:18:51.698758 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.698660    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf" event={"ID":"17ba2a01-0007-4f37-b1a9-464910fd82e6","Type":"ContainerDied","Data":"a557ecfed49b1f9da927736ecd1eb28440a36b8c36e15ed5cd3573e3e2b69b9a"}
Oct 30 19:18:51.698758 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.698704    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf" event={"ID":"17ba2a01-0007-4f37-b1a9-464910fd82e6","Type":"ContainerDied","Data":"ee1360cb715a95231c44d5aa78a9803448cdc239c47bb54ffdb081358107efcd"}
Oct 30 19:18:51.698758 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.698726    5820 scope.go:122] "RemoveContainer" containerID="a557ecfed49b1f9da927736ecd1eb28440a36b8c36e15ed5cd3573e3e2b69b9a"
Oct 30 19:18:51.698758 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.698663    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"
Oct 30 19:18:51.709044 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.709025    5820 scope.go:122] "RemoveContainer" containerID="b38c1203a7f746311664f4a5481d28f31e69bb32071a0d451a1670d375f31cd8"
Oct 30 19:18:51.718011 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.717995    5820 scope.go:122] "RemoveContainer" containerID="a557ecfed49b1f9da927736ecd1eb28440a36b8c36e15ed5cd3573e3e2b69b9a"
Oct 30 19:18:51.718294 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:18:51.718272    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a557ecfed49b1f9da927736ecd1eb28440a36b8c36e15ed5cd3573e3e2b69b9a\": container with ID starting with a557ecfed49b1f9da927736ecd1eb28440a36b8c36e15ed5cd3573e3e2b69b9a not found: ID does not exist" containerID="a557ecfed49b1f9da927736ecd1eb28440a36b8c36e15ed5cd3573e3e2b69b9a"
Oct 30 19:18:51.718339 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.718303    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a557ecfed49b1f9da927736ecd1eb28440a36b8c36e15ed5cd3573e3e2b69b9a"} err="failed to get container status \"a557ecfed49b1f9da927736ecd1eb28440a36b8c36e15ed5cd3573e3e2b69b9a\": rpc error: code = NotFound desc = could not find container \"a557ecfed49b1f9da927736ecd1eb28440a36b8c36e15ed5cd3573e3e2b69b9a\": container with ID starting with a557ecfed49b1f9da927736ecd1eb28440a36b8c36e15ed5cd3573e3e2b69b9a not found: ID does not exist"
Oct 30 19:18:51.718339 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.718322    5820 scope.go:122] "RemoveContainer" containerID="b38c1203a7f746311664f4a5481d28f31e69bb32071a0d451a1670d375f31cd8"
Oct 30 19:18:51.718556 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:18:51.718537    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b38c1203a7f746311664f4a5481d28f31e69bb32071a0d451a1670d375f31cd8\": container with ID starting with b38c1203a7f746311664f4a5481d28f31e69bb32071a0d451a1670d375f31cd8 not found: ID does not exist" containerID="b38c1203a7f746311664f4a5481d28f31e69bb32071a0d451a1670d375f31cd8"
Oct 30 19:18:51.718628 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.718567    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b38c1203a7f746311664f4a5481d28f31e69bb32071a0d451a1670d375f31cd8"} err="failed to get container status \"b38c1203a7f746311664f4a5481d28f31e69bb32071a0d451a1670d375f31cd8\": rpc error: code = NotFound desc = could not find container \"b38c1203a7f746311664f4a5481d28f31e69bb32071a0d451a1670d375f31cd8\": container with ID starting with b38c1203a7f746311664f4a5481d28f31e69bb32071a0d451a1670d375f31cd8 not found: ID does not exist"
Oct 30 19:18:51.865462 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.865430    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"]
Oct 30 19:18:51.873658 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:51.873629    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-91cb32-openshift-default-68b485b949-xmhwf"]
Oct 30 19:18:53.436499 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:53.436466    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="17ba2a01-0007-4f37-b1a9-464910fd82e6" path="/var/lib/kubelet/pods/17ba2a01-0007-4f37-b1a9-464910fd82e6/volumes"
Oct 30 19:18:53.987195 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:53.987159    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:54.173092 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.173048    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:54.507973 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.505075    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd"]
Oct 30 19:18:54.507973 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.506135    5820 cpu_manager.go:414] "containerMap: removing container" podUID="17ba2a01-0007-4f37-b1a9-464910fd82e6" containerName="istio-proxy"
Oct 30 19:18:54.507973 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.506161    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17ba2a01-0007-4f37-b1a9-464910fd82e6" containerName="istio-proxy"
Oct 30 19:18:54.507973 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.506213    5820 cpu_manager.go:414] "containerMap: removing container" podUID="17ba2a01-0007-4f37-b1a9-464910fd82e6" containerName="istio-proxy"
Oct 30 19:18:54.507973 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.506223    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17ba2a01-0007-4f37-b1a9-464910fd82e6" containerName="istio-proxy"
Oct 30 19:18:54.507973 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.506453    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="17ba2a01-0007-4f37-b1a9-464910fd82e6" containerName="istio-proxy"
Oct 30 19:18:54.507973 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.506475    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="17ba2a01-0007-4f37-b1a9-464910fd82e6" containerName="istio-proxy"
Oct 30 19:18:54.519977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.514310    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd"
Oct 30 19:18:54.580203 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.577730    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd"]
Oct 30 19:18:54.585651 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.585619    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dglj\" (UniqueName: \"kubernetes.io/projected/349f7a4a-cae3-41b7-a6dd-ead82d9b0a64-kube-api-access-2dglj\") pod \"payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd\" (UID: \"349f7a4a-cae3-41b7-a6dd-ead82d9b0a64\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd"
Oct 30 19:18:54.588984 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.587044    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/349f7a4a-cae3-41b7-a6dd-ead82d9b0a64-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd\" (UID: \"349f7a4a-cae3-41b7-a6dd-ead82d9b0a64\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd"
Oct 30 19:18:54.615986 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.615943    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-8f9c6b-dockercfg-2nflm\""
Oct 30 19:18:54.616136 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.616064    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-8f9c6b\""
Oct 30 19:18:54.674919 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.674887    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74"]
Oct 30 19:18:54.679174 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.679147    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74"
Oct 30 19:18:54.690257 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.689286    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/349f7a4a-cae3-41b7-a6dd-ead82d9b0a64-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd\" (UID: \"349f7a4a-cae3-41b7-a6dd-ead82d9b0a64\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd"
Oct 30 19:18:54.690257 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.689445    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dglj\" (UniqueName: \"kubernetes.io/projected/349f7a4a-cae3-41b7-a6dd-ead82d9b0a64-kube-api-access-2dglj\") pod \"payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd\" (UID: \"349f7a4a-cae3-41b7-a6dd-ead82d9b0a64\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd"
Oct 30 19:18:54.690831 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.690797    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/349f7a4a-cae3-41b7-a6dd-ead82d9b0a64-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd\" (UID: \"349f7a4a-cae3-41b7-a6dd-ead82d9b0a64\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd"
Oct 30 19:18:54.737507 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.737468    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74"]
Oct 30 19:18:54.751829 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.751796    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dglj\" (UniqueName: \"kubernetes.io/projected/349f7a4a-cae3-41b7-a6dd-ead82d9b0a64-kube-api-access-2dglj\") pod \"payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd\" (UID: \"349f7a4a-cae3-41b7-a6dd-ead82d9b0a64\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd"
Oct 30 19:18:54.790581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.790498    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfmdz\" (UniqueName: \"kubernetes.io/projected/336a57c2-7270-4cd3-843f-b8a63a26bcab-kube-api-access-mfmdz\") pod \"payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74\" (UID: \"336a57c2-7270-4cd3-843f-b8a63a26bcab\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74"
Oct 30 19:18:54.790730 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.790582    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/336a57c2-7270-4cd3-843f-b8a63a26bcab-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74\" (UID: \"336a57c2-7270-4cd3-843f-b8a63a26bcab\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74"
Oct 30 19:18:54.854402 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.854367    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd"
Oct 30 19:18:54.891552 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.891517    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfmdz\" (UniqueName: \"kubernetes.io/projected/336a57c2-7270-4cd3-843f-b8a63a26bcab-kube-api-access-mfmdz\") pod \"payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74\" (UID: \"336a57c2-7270-4cd3-843f-b8a63a26bcab\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74"
Oct 30 19:18:54.891712 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.891579    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/336a57c2-7270-4cd3-843f-b8a63a26bcab-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74\" (UID: \"336a57c2-7270-4cd3-843f-b8a63a26bcab\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74"
Oct 30 19:18:54.892328 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.892305    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/336a57c2-7270-4cd3-843f-b8a63a26bcab-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74\" (UID: \"336a57c2-7270-4cd3-843f-b8a63a26bcab\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74"
Oct 30 19:18:54.954003 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.953264    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfmdz\" (UniqueName: \"kubernetes.io/projected/336a57c2-7270-4cd3-843f-b8a63a26bcab-kube-api-access-mfmdz\") pod \"payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74\" (UID: \"336a57c2-7270-4cd3-843f-b8a63a26bcab\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74"
Oct 30 19:18:54.997868 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:54.997833    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74"
Oct 30 19:18:55.245795 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:55.245767    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd"]
Oct 30 19:18:55.247097 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:18:55.247068    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod349f7a4a_cae3_41b7_a6dd_ead82d9b0a64.slice/crio-e28321a205e48d0d139305d27824d5207d4086a207ea61f0633c097f7fabaf88 WatchSource:0}: Error finding container e28321a205e48d0d139305d27824d5207d4086a207ea61f0633c097f7fabaf88: Status 404 returned error can't find the container with id e28321a205e48d0d139305d27824d5207d4086a207ea61f0633c097f7fabaf88
Oct 30 19:18:55.375103 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:55.375080    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74"]
Oct 30 19:18:55.377102 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:18:55.377073    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod336a57c2_7270_4cd3_843f_b8a63a26bcab.slice/crio-951742578cf402ac5b0684c959ebd3e2e14abefc374022b4c9b735eafa8d5d4d WatchSource:0}: Error finding container 951742578cf402ac5b0684c959ebd3e2e14abefc374022b4c9b735eafa8d5d4d: Status 404 returned error can't find the container with id 951742578cf402ac5b0684c959ebd3e2e14abefc374022b4c9b735eafa8d5d4d
Oct 30 19:18:55.724497 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:55.724404    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74" event={"ID":"336a57c2-7270-4cd3-843f-b8a63a26bcab","Type":"ContainerStarted","Data":"02be29a76ccecb6927fb8f06b7a5c8407e9f75500d93041668d5714dcce42387"}
Oct 30 19:18:55.724497 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:55.724456    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74" event={"ID":"336a57c2-7270-4cd3-843f-b8a63a26bcab","Type":"ContainerStarted","Data":"951742578cf402ac5b0684c959ebd3e2e14abefc374022b4c9b735eafa8d5d4d"}
Oct 30 19:18:55.725014 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:55.724560    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74"
Oct 30 19:18:55.726092 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:55.726069    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd" event={"ID":"349f7a4a-cae3-41b7-a6dd-ead82d9b0a64","Type":"ContainerStarted","Data":"99f87264fe6edfd2398466ae9afa612a733bfcd7c1b5d80b99eac2d288db4c27"}
Oct 30 19:18:55.726200 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:55.726108    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd"
Oct 30 19:18:55.726200 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:55.726124    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd" event={"ID":"349f7a4a-cae3-41b7-a6dd-ead82d9b0a64","Type":"ContainerStarted","Data":"e28321a205e48d0d139305d27824d5207d4086a207ea61f0633c097f7fabaf88"}
Oct 30 19:18:55.842183 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:55.842134    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74" podStartSLOduration=1.8421205600000001 podStartE2EDuration="1.84212056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:55.839483739 +0000 UTC m=+2656.551232436" watchObservedRunningTime="2026-10-30 19:18:55.84212056 +0000 UTC m=+2656.553869250"
Oct 30 19:18:55.983391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:55.981475    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd" podStartSLOduration=1.9814572620000002 podStartE2EDuration="1.981457262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:55.977445783 +0000 UTC m=+2656.689194540" watchObservedRunningTime="2026-10-30 19:18:55.981457262 +0000 UTC m=+2656.693205961"
Oct 30 19:18:56.232237 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:56.232207    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:56.473978 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:56.470819    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:56.656020 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:56.655984    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:57.746620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:57.746585    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:57.990163 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:57.990122    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:58.397007 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.396929    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:58.587112 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.587080    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"]
Oct 30 19:18:58.591761 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.591737    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.595455 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.595407    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:58.720032 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.715116    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w0-19f6ea-openshift-default-dockercfg-vs94r\""
Oct 30 19:18:58.720979 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.720501    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"]
Oct 30 19:18:58.741971 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.738756    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-data\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.741971 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.738833    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-token\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.741971 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.738875    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1720171-99ee-4be0-8bbe-f8fdfed25319-istiod-ca-cert\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.741971 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.738935    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-workload-socket\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.741971 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.739009    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-envoy\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.741971 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.739089    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-credential-socket\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.741971 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.739119    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmxnt\" (UniqueName: \"kubernetes.io/projected/c1720171-99ee-4be0-8bbe-f8fdfed25319-kube-api-access-nmxnt\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.741971 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.739168    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-podinfo\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.741971 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.739263    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-workload-certs\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.834358 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.834319    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:58.843729 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.842214    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-token\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.849191 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.849150    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1720171-99ee-4be0-8bbe-f8fdfed25319-istiod-ca-cert\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.849384 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.849261    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-workload-socket\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.849384 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.849336    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-envoy\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.849537 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.849447    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-credential-socket\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.849537 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.849485    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmxnt\" (UniqueName: \"kubernetes.io/projected/c1720171-99ee-4be0-8bbe-f8fdfed25319-kube-api-access-nmxnt\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.849661 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.849538    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-podinfo\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.849784 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.849705    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-workload-certs\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.849857 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.849812    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-data\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.850111 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.850086    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-workload-socket\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.854102 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.854074    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-workload-certs\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.854346 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.854326    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-data\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.854452 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.854433    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-credential-socket\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.855089 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.855065    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1720171-99ee-4be0-8bbe-f8fdfed25319-istiod-ca-cert\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.855566 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.855540    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-podinfo\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.857676 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.857628    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-envoy\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.913105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.913069    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmxnt\" (UniqueName: \"kubernetes.io/projected/c1720171-99ee-4be0-8bbe-f8fdfed25319-kube-api-access-nmxnt\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:58.914007 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:58.913985    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-token\") pod \"e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") " pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:59.091839 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:59.091788    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:59.206371 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:59.206338    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:18:59.542840 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:59.542747    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:59.658273 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:59.658242    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"]
Oct 30 19:18:59.659560 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:18:59.659529    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1720171_99ee_4be0_8bbe_f8fdfed25319.slice/crio-62373eaa6dc8fda763c1369633eea1fd6b9f06f227cf507d964367ecdd492c2d WatchSource:0}: Error finding container 62373eaa6dc8fda763c1369633eea1fd6b9f06f227cf507d964367ecdd492c2d: Status 404 returned error can't find the container with id 62373eaa6dc8fda763c1369633eea1fd6b9f06f227cf507d964367ecdd492c2d
Oct 30 19:18:59.662725 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:59.662680    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:18:59.662807 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:59.662778    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:18:59.662864 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:59.662825    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:18:59.829426 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:59.829387    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9" event={"ID":"c1720171-99ee-4be0-8bbe-f8fdfed25319","Type":"ContainerStarted","Data":"8f6b68d55087a853db1204e07278fe0fa7cba762fed7fd73f9087270a41675d7"}
Oct 30 19:18:59.829594 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:59.829457    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9" event={"ID":"c1720171-99ee-4be0-8bbe-f8fdfed25319","Type":"ContainerStarted","Data":"62373eaa6dc8fda763c1369633eea1fd6b9f06f227cf507d964367ecdd492c2d"}
Oct 30 19:18:59.958242 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:18:59.958190    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9" podStartSLOduration=1.958177004 podStartE2EDuration="1.958177004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:59.956745268 +0000 UTC m=+2660.668493964" watchObservedRunningTime="2026-10-30 19:18:59.958177004 +0000 UTC m=+2660.669925701"
Oct 30 19:19:00.206703 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:00.206624    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:19:00.212231 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:00.212208    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:19:00.836339 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:00.836305    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:19:00.837484 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:00.837460    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:19:01.573978 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:01.571601    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:01.790985 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:01.789570    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:01.868079 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:01.867992    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw"]
Oct 30 19:19:01.872232 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:01.872211    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw"
Oct 30 19:19:01.925978 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:01.925925    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w1-23302c-dockercfg-xzt8c\""
Oct 30 19:19:01.929963 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:01.929922    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w1-23302c\""
Oct 30 19:19:01.933462 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:01.933437    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw"]
Oct 30 19:19:02.008179 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:02.008140    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7c4k\" (UniqueName: \"kubernetes.io/projected/bf3c4c7a-8bb7-440f-8d63-1d961dca8371-kube-api-access-m7c4k\") pod \"payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw\" (UID: \"bf3c4c7a-8bb7-440f-8d63-1d961dca8371\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw"
Oct 30 19:19:02.008638 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:02.008616    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf3c4c7a-8bb7-440f-8d63-1d961dca8371-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw\" (UID: \"bf3c4c7a-8bb7-440f-8d63-1d961dca8371\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw"
Oct 30 19:19:02.110142 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:02.110108    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf3c4c7a-8bb7-440f-8d63-1d961dca8371-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw\" (UID: \"bf3c4c7a-8bb7-440f-8d63-1d961dca8371\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw"
Oct 30 19:19:02.110347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:02.110188    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7c4k\" (UniqueName: \"kubernetes.io/projected/bf3c4c7a-8bb7-440f-8d63-1d961dca8371-kube-api-access-m7c4k\") pod \"payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw\" (UID: \"bf3c4c7a-8bb7-440f-8d63-1d961dca8371\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw"
Oct 30 19:19:02.110822 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:02.110799    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf3c4c7a-8bb7-440f-8d63-1d961dca8371-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw\" (UID: \"bf3c4c7a-8bb7-440f-8d63-1d961dca8371\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw"
Oct 30 19:19:02.171408 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:02.171336    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7c4k\" (UniqueName: \"kubernetes.io/projected/bf3c4c7a-8bb7-440f-8d63-1d961dca8371-kube-api-access-m7c4k\") pod \"payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw\" (UID: \"bf3c4c7a-8bb7-440f-8d63-1d961dca8371\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw"
Oct 30 19:19:02.188261 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:02.188233    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw"
Oct 30 19:19:02.572578 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:02.572550    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw"]
Oct 30 19:19:02.574631 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:19:02.574603    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf3c4c7a_8bb7_440f_8d63_1d961dca8371.slice/crio-7b5dcf4b1e985fd03e5f7f6a77b420cdda7946001c3a8a0f1e7bbee28d51255b WatchSource:0}: Error finding container 7b5dcf4b1e985fd03e5f7f6a77b420cdda7946001c3a8a0f1e7bbee28d51255b: Status 404 returned error can't find the container with id 7b5dcf4b1e985fd03e5f7f6a77b420cdda7946001c3a8a0f1e7bbee28d51255b
Oct 30 19:19:02.848934 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:02.848840    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw" event={"ID":"bf3c4c7a-8bb7-440f-8d63-1d961dca8371","Type":"ContainerStarted","Data":"e7b981d4fc6545892f11d5ea9d889e78e423b5e5c11e9e722f232a2e55768cd0"}
Oct 30 19:19:02.848934 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:02.848897    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw" event={"ID":"bf3c4c7a-8bb7-440f-8d63-1d961dca8371","Type":"ContainerStarted","Data":"7b5dcf4b1e985fd03e5f7f6a77b420cdda7946001c3a8a0f1e7bbee28d51255b"}
Oct 30 19:19:02.849170 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:02.849135    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw"
Oct 30 19:19:02.964749 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:02.964694    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw" podStartSLOduration=1.964676441 podStartE2EDuration="1.964676441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:02.964021477 +0000 UTC m=+2663.675770170" watchObservedRunningTime="2026-10-30 19:19:02.964676441 +0000 UTC m=+2663.676425134"
Oct 30 19:19:04.015866 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.015829    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:04.239783 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.239741    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:04.399719 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.399686    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n"]
Oct 30 19:19:04.403595 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.403576    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n"
Oct 30 19:19:04.435267 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.435238    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:04.437372 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.437349    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p45vr\" (UniqueName: \"kubernetes.io/projected/dba918a1-39a1-41ad-bb05-55a5c5f6981c-kube-api-access-p45vr\") pod \"payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n\" (UID: \"dba918a1-39a1-41ad-bb05-55a5c5f6981c\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n"
Oct 30 19:19:04.437447 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.437423    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dba918a1-39a1-41ad-bb05-55a5c5f6981c-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n\" (UID: \"dba918a1-39a1-41ad-bb05-55a5c5f6981c\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n"
Oct 30 19:19:04.461749 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.461722    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n"]
Oct 30 19:19:04.508718 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.508687    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-a-81d66e-dockercfg-7s5dg\""
Oct 30 19:19:04.515036 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.509005    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-a-81d66e\""
Oct 30 19:19:04.545887 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.545848    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p45vr\" (UniqueName: \"kubernetes.io/projected/dba918a1-39a1-41ad-bb05-55a5c5f6981c-kube-api-access-p45vr\") pod \"payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n\" (UID: \"dba918a1-39a1-41ad-bb05-55a5c5f6981c\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n"
Oct 30 19:19:04.546158 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.545989    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dba918a1-39a1-41ad-bb05-55a5c5f6981c-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n\" (UID: \"dba918a1-39a1-41ad-bb05-55a5c5f6981c\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n"
Oct 30 19:19:04.570087 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.548051    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dba918a1-39a1-41ad-bb05-55a5c5f6981c-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n\" (UID: \"dba918a1-39a1-41ad-bb05-55a5c5f6981c\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n"
Oct 30 19:19:04.570087 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.567989    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc"]
Oct 30 19:19:04.579916 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.579889    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc"
Oct 30 19:19:04.607839 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.607813    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p45vr\" (UniqueName: \"kubernetes.io/projected/dba918a1-39a1-41ad-bb05-55a5c5f6981c-kube-api-access-p45vr\") pod \"payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n\" (UID: \"dba918a1-39a1-41ad-bb05-55a5c5f6981c\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n"
Oct 30 19:19:04.612893 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.612863    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc"]
Oct 30 19:19:04.621242 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.620984    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:04.647440 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.647407    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dbe29c1-0a26-4859-a57c-460ddc518f24-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc\" (UID: \"9dbe29c1-0a26-4859-a57c-460ddc518f24\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc"
Oct 30 19:19:04.647608 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.647520    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q558k\" (UniqueName: \"kubernetes.io/projected/9dbe29c1-0a26-4859-a57c-460ddc518f24-kube-api-access-q558k\") pod \"payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc\" (UID: \"9dbe29c1-0a26-4859-a57c-460ddc518f24\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc"
Oct 30 19:19:04.717291 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.717212    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n"
Oct 30 19:19:04.748780 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.748745    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dbe29c1-0a26-4859-a57c-460ddc518f24-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc\" (UID: \"9dbe29c1-0a26-4859-a57c-460ddc518f24\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc"
Oct 30 19:19:04.748987 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.748815    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q558k\" (UniqueName: \"kubernetes.io/projected/9dbe29c1-0a26-4859-a57c-460ddc518f24-kube-api-access-q558k\") pod \"payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc\" (UID: \"9dbe29c1-0a26-4859-a57c-460ddc518f24\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc"
Oct 30 19:19:04.749564 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.749544    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dbe29c1-0a26-4859-a57c-460ddc518f24-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc\" (UID: \"9dbe29c1-0a26-4859-a57c-460ddc518f24\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc"
Oct 30 19:19:04.825023 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.823980    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:04.854418 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.854385    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q558k\" (UniqueName: \"kubernetes.io/projected/9dbe29c1-0a26-4859-a57c-460ddc518f24-kube-api-access-q558k\") pod \"payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc\" (UID: \"9dbe29c1-0a26-4859-a57c-460ddc518f24\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc"
Oct 30 19:19:04.896673 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:04.896643    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc"
Oct 30 19:19:05.097086 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:05.097050    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:05.108141 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:05.108113    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n"]
Oct 30 19:19:05.109715 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:19:05.109688    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba918a1_39a1_41ad_bb05_55a5c5f6981c.slice/crio-44790806b242db41bb272d7623faf8296c6419ecf875dc498c480c653aced312 WatchSource:0}: Error finding container 44790806b242db41bb272d7623faf8296c6419ecf875dc498c480c653aced312: Status 404 returned error can't find the container with id 44790806b242db41bb272d7623faf8296c6419ecf875dc498c480c653aced312
Oct 30 19:19:05.283356 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:05.283324    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc"]
Oct 30 19:19:05.285164 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:19:05.285140    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dbe29c1_0a26_4859_a57c_460ddc518f24.slice/crio-c85c58cf9884e6546b39645edd2ce6bb4ca846e8c90df73c4f7e4686e1e519ff WatchSource:0}: Error finding container c85c58cf9884e6546b39645edd2ce6bb4ca846e8c90df73c4f7e4686e1e519ff: Status 404 returned error can't find the container with id c85c58cf9884e6546b39645edd2ce6bb4ca846e8c90df73c4f7e4686e1e519ff
Oct 30 19:19:05.315760 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:05.315727    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:05.503278 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:05.503182    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:05.877562 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:05.877519    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc" event={"ID":"9dbe29c1-0a26-4859-a57c-460ddc518f24","Type":"ContainerStarted","Data":"d5f0aa1d66542cfe2a59560d13635810afa31971f898cbe33506c341a4b8bd95"}
Oct 30 19:19:05.877785 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:05.877600    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc"
Oct 30 19:19:05.877785 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:05.877627    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc" event={"ID":"9dbe29c1-0a26-4859-a57c-460ddc518f24","Type":"ContainerStarted","Data":"c85c58cf9884e6546b39645edd2ce6bb4ca846e8c90df73c4f7e4686e1e519ff"}
Oct 30 19:19:05.879164 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:05.879135    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n" event={"ID":"dba918a1-39a1-41ad-bb05-55a5c5f6981c","Type":"ContainerStarted","Data":"674640f6e10ac4d5ba3d303774ea5e7dc2398714504ff0c435bfbacd59ecdbac"}
Oct 30 19:19:05.879295 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:05.879185    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n" event={"ID":"dba918a1-39a1-41ad-bb05-55a5c5f6981c","Type":"ContainerStarted","Data":"44790806b242db41bb272d7623faf8296c6419ecf875dc498c480c653aced312"}
Oct 30 19:19:05.879295 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:05.879279    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n"
Oct 30 19:19:06.042646 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:06.042596    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc" podStartSLOduration=2.042583616 podStartE2EDuration="2.042583616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:05.992679462 +0000 UTC m=+2666.704428159" watchObservedRunningTime="2026-10-30 19:19:06.042583616 +0000 UTC m=+2666.754332312"
Oct 30 19:19:06.115547 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:06.115185    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n" podStartSLOduration=2.115164486 podStartE2EDuration="2.115164486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:06.112362496 +0000 UTC m=+2666.824111194" watchObservedRunningTime="2026-10-30 19:19:06.115164486 +0000 UTC m=+2666.826913184"
Oct 30 19:19:06.736561 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:06.736528    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd"
Oct 30 19:19:06.736787 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:06.736597    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74"
Oct 30 19:19:08.204208 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:19:08.204168    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:19:09.939131 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:09.939094    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:13.748045 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:13.748009    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:13.858620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:13.858592    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw"
Oct 30 19:19:14.267260 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:14.267202    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp"]
Oct 30 19:19:14.278575 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:14.278546    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp"
Oct 30 19:19:14.311733 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:14.311691    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:14.320910 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:14.320880    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp"]
Oct 30 19:19:14.331088 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:14.331060    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-b-9f939e\""
Oct 30 19:19:14.331293 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:14.331190    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-b-9f939e-dockercfg-xdfnz\""
Oct 30 19:19:14.384646 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:14.384613    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88a930a8-b6ff-4b13-a9d0-6e4728e55360-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp\" (UID: \"88a930a8-b6ff-4b13-a9d0-6e4728e55360\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp"
Oct 30 19:19:14.384845 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:14.384684    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssznw\" (UniqueName: \"kubernetes.io/projected/88a930a8-b6ff-4b13-a9d0-6e4728e55360-kube-api-access-ssznw\") pod \"payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp\" (UID: \"88a930a8-b6ff-4b13-a9d0-6e4728e55360\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp"
Oct 30 19:19:14.485489 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:14.485446    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88a930a8-b6ff-4b13-a9d0-6e4728e55360-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp\" (UID: \"88a930a8-b6ff-4b13-a9d0-6e4728e55360\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp"
Oct 30 19:19:14.485664 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:14.485547    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssznw\" (UniqueName: \"kubernetes.io/projected/88a930a8-b6ff-4b13-a9d0-6e4728e55360-kube-api-access-ssznw\") pod \"payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp\" (UID: \"88a930a8-b6ff-4b13-a9d0-6e4728e55360\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp"
Oct 30 19:19:14.486245 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:14.486226    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88a930a8-b6ff-4b13-a9d0-6e4728e55360-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp\" (UID: \"88a930a8-b6ff-4b13-a9d0-6e4728e55360\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp"
Oct 30 19:19:14.546524 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:14.546444    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssznw\" (UniqueName: \"kubernetes.io/projected/88a930a8-b6ff-4b13-a9d0-6e4728e55360-kube-api-access-ssznw\") pod \"payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp\" (UID: \"88a930a8-b6ff-4b13-a9d0-6e4728e55360\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp"
Oct 30 19:19:14.605583 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:14.605551    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp"
Oct 30 19:19:14.787137 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:14.786200    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:14.986250 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:14.986224    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp"]
Oct 30 19:19:14.987475 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:19:14.987450    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88a930a8_b6ff_4b13_a9d0_6e4728e55360.slice/crio-1a7d94fc6320af4f14739d52ad17f256ec4196db00cfe453542d129026941592 WatchSource:0}: Error finding container 1a7d94fc6320af4f14739d52ad17f256ec4196db00cfe453542d129026941592: Status 404 returned error can't find the container with id 1a7d94fc6320af4f14739d52ad17f256ec4196db00cfe453542d129026941592
Oct 30 19:19:15.949450 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:15.949416    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp" event={"ID":"88a930a8-b6ff-4b13-a9d0-6e4728e55360","Type":"ContainerStarted","Data":"f2203a5ef3a85f9c4e0cc99806f4eaad3c1de6a132e9744284a442303f708008"}
Oct 30 19:19:15.949944 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:15.949470    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp" event={"ID":"88a930a8-b6ff-4b13-a9d0-6e4728e55360","Type":"ContainerStarted","Data":"1a7d94fc6320af4f14739d52ad17f256ec4196db00cfe453542d129026941592"}
Oct 30 19:19:15.949944 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:15.949632    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp"
Oct 30 19:19:16.072102 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:16.072036    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp" podStartSLOduration=2.072018363 podStartE2EDuration="2.072018363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:16.071209257 +0000 UTC m=+2676.782957955" watchObservedRunningTime="2026-10-30 19:19:16.072018363 +0000 UTC m=+2676.783767063"
Oct 30 19:19:16.354519 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:16.354481    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:16.549439 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:16.549405    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:16.925191 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:16.924131    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc"
Oct 30 19:19:16.926322 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:16.925625    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n"
Oct 30 19:19:19.651779 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:19.651741    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:24.591626 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:24.591584    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:26.962618 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:26.962591    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp"
Oct 30 19:19:28.999655 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:28.999616    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"]
Oct 30 19:19:29.005039 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:29.005020    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"
Oct 30 19:19:29.071441 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:29.071400    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/612e60bf-47da-407a-b30d-6213966bd7a9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp\" (UID: \"612e60bf-47da-407a-b30d-6213966bd7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"
Oct 30 19:19:29.071618 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:29.071523    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/612e60bf-47da-407a-b30d-6213966bd7a9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp\" (UID: \"612e60bf-47da-407a-b30d-6213966bd7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"
Oct 30 19:19:29.071618 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:29.071599    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spk2n\" (UniqueName: \"kubernetes.io/projected/612e60bf-47da-407a-b30d-6213966bd7a9-kube-api-access-spk2n\") pod \"maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp\" (UID: \"612e60bf-47da-407a-b30d-6213966bd7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"
Oct 30 19:19:29.071719 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:29.071661    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/612e60bf-47da-407a-b30d-6213966bd7a9-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp\" (UID: \"612e60bf-47da-407a-b30d-6213966bd7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"
Oct 30 19:19:29.109205 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:29.109175    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-8f9c6b-serving-cert\""
Oct 30 19:19:29.111810 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:29.111788    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"]
Oct 30 19:19:29.172932 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:29.172902    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/612e60bf-47da-407a-b30d-6213966bd7a9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp\" (UID: \"612e60bf-47da-407a-b30d-6213966bd7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"
Oct 30 19:19:29.173140 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:29.173015    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/612e60bf-47da-407a-b30d-6213966bd7a9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp\" (UID: \"612e60bf-47da-407a-b30d-6213966bd7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"
Oct 30 19:19:29.173140 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:29.173067    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spk2n\" (UniqueName: \"kubernetes.io/projected/612e60bf-47da-407a-b30d-6213966bd7a9-kube-api-access-spk2n\") pod \"maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp\" (UID: \"612e60bf-47da-407a-b30d-6213966bd7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"
Oct 30 19:19:29.173265 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:29.173161    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/612e60bf-47da-407a-b30d-6213966bd7a9-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp\" (UID: \"612e60bf-47da-407a-b30d-6213966bd7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"
Oct 30 19:19:29.173265 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:19:29.173178    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-serving-cert: secret "maas-api-e2e-shared-a-w2-8f9c6b-serving-cert" not found
Oct 30 19:19:29.173265 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:19:29.173257    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/612e60bf-47da-407a-b30d-6213966bd7a9-maas-api-tls podName:612e60bf-47da-407a-b30d-6213966bd7a9 nodeName:}" failed. No retries permitted until 2026-10-30 19:19:29.673234234 +0000 UTC m=+2690.384982923 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/612e60bf-47da-407a-b30d-6213966bd7a9-maas-api-tls") pod "maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp" (UID: "612e60bf-47da-407a-b30d-6213966bd7a9") : secret "maas-api-e2e-shared-a-w2-8f9c6b-serving-cert" not found
Oct 30 19:19:29.173610 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:29.173592    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/612e60bf-47da-407a-b30d-6213966bd7a9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp\" (UID: \"612e60bf-47da-407a-b30d-6213966bd7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"
Oct 30 19:19:29.175742 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:29.175716    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/612e60bf-47da-407a-b30d-6213966bd7a9-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp\" (UID: \"612e60bf-47da-407a-b30d-6213966bd7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"
Oct 30 19:19:29.236572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:29.236548    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spk2n\" (UniqueName: \"kubernetes.io/projected/612e60bf-47da-407a-b30d-6213966bd7a9-kube-api-access-spk2n\") pod \"maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp\" (UID: \"612e60bf-47da-407a-b30d-6213966bd7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"
Oct 30 19:19:29.237482 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:29.237457    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:29.456621 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:29.456564    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:29.678657 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:29.678626    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/612e60bf-47da-407a-b30d-6213966bd7a9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp\" (UID: \"612e60bf-47da-407a-b30d-6213966bd7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"
Oct 30 19:19:29.681142 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:29.681121    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/612e60bf-47da-407a-b30d-6213966bd7a9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp\" (UID: \"612e60bf-47da-407a-b30d-6213966bd7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"
Oct 30 19:19:29.918453 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:29.918421    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"
Oct 30 19:19:30.304780 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:30.304753    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"]
Oct 30 19:19:30.305220 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:19:30.305079    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod612e60bf_47da_407a_b30d_6213966bd7a9.slice/crio-d47cf82a10489c3b8ea920a3e86c86d3bdb415351e52cab809fbbc65f5db3568 WatchSource:0}: Error finding container d47cf82a10489c3b8ea920a3e86c86d3bdb415351e52cab809fbbc65f5db3568: Status 404 returned error can't find the container with id d47cf82a10489c3b8ea920a3e86c86d3bdb415351e52cab809fbbc65f5db3568
Oct 30 19:19:31.045385 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:31.045347    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp" event={"ID":"612e60bf-47da-407a-b30d-6213966bd7a9","Type":"ContainerStarted","Data":"d47cf82a10489c3b8ea920a3e86c86d3bdb415351e52cab809fbbc65f5db3568"}
Oct 30 19:19:32.092974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:32.083415    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp" event={"ID":"612e60bf-47da-407a-b30d-6213966bd7a9","Type":"ContainerStarted","Data":"fb9539f54fdf77b6412d22fb1e11e2f443dd93cdb749f0776df49ed7b77f18b6"}
Oct 30 19:19:32.092974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:32.084455    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"
Oct 30 19:19:32.203287 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:32.203233    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp" podStartSLOduration=3.3190989379999998 podStartE2EDuration="4.203219269s" totalImagesPullingTime="884.120331ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:32.202285713 +0000 UTC m=+2692.914034409" watchObservedRunningTime="2026-10-30 19:19:32.203219269 +0000 UTC m=+2692.914967966"
Oct 30 19:19:33.933363 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:33.933329    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"]
Oct 30 19:19:33.937338 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:33.937321    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"
Oct 30 19:19:33.989546 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:33.989513    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w1-23302c-serving-cert\""
Oct 30 19:19:34.031174 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:34.031146    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5\" (UID: \"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"
Oct 30 19:19:34.031345 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:34.031258    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5\" (UID: \"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"
Oct 30 19:19:34.031345 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:34.031334    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrs9c\" (UniqueName: \"kubernetes.io/projected/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-kube-api-access-vrs9c\") pod \"maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5\" (UID: \"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"
Oct 30 19:19:34.031430 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:34.031360    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5\" (UID: \"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"
Oct 30 19:19:34.041748 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:34.041722    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"]
Oct 30 19:19:34.132847 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:34.132807    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5\" (UID: \"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"
Oct 30 19:19:34.133090 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:34.132911    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5\" (UID: \"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"
Oct 30 19:19:34.133090 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:34.133027    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrs9c\" (UniqueName: \"kubernetes.io/projected/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-kube-api-access-vrs9c\") pod \"maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5\" (UID: \"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"
Oct 30 19:19:34.133090 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:19:34.133046    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-serving-cert: secret "maas-api-e2e-shared-a-w1-23302c-serving-cert" not found
Oct 30 19:19:34.133090 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:34.133067    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5\" (UID: \"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"
Oct 30 19:19:34.133284 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:19:34.133122    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-maas-api-tls podName:18dfcf17-bc5a-43d9-aff6-4cc57d2a1233 nodeName:}" failed. No retries permitted until 2026-10-30 19:19:34.633101664 +0000 UTC m=+2695.344850354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-maas-api-tls") pod "maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5" (UID: "18dfcf17-bc5a-43d9-aff6-4cc57d2a1233") : secret "maas-api-e2e-shared-a-w1-23302c-serving-cert" not found
Oct 30 19:19:34.133692 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:34.133673    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5\" (UID: \"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"
Oct 30 19:19:34.135511 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:34.135493    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5\" (UID: \"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"
Oct 30 19:19:34.169814 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:34.169783    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:34.220967 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:34.220861    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrs9c\" (UniqueName: \"kubernetes.io/projected/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-kube-api-access-vrs9c\") pod \"maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5\" (UID: \"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"
Oct 30 19:19:34.359606 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:34.359531    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:34.540242 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:34.540152    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:34.652818 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:34.652782    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5\" (UID: \"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"
Oct 30 19:19:34.655443 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:34.655423    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5\" (UID: \"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"
Oct 30 19:19:34.851527 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:34.851505    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"
Oct 30 19:19:35.235010 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:35.234986    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"]
Oct 30 19:19:35.236635 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:19:35.236595    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18dfcf17_bc5a_43d9_aff6_4cc57d2a1233.slice/crio-ad2d7acd72db360825371bb7977ff9679ffcc09dbf435cbaf2e3c32afb6bf7ad WatchSource:0}: Error finding container ad2d7acd72db360825371bb7977ff9679ffcc09dbf435cbaf2e3c32afb6bf7ad: Status 404 returned error can't find the container with id ad2d7acd72db360825371bb7977ff9679ffcc09dbf435cbaf2e3c32afb6bf7ad
Oct 30 19:19:36.110240 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:36.110206    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5" event={"ID":"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233","Type":"ContainerStarted","Data":"ad2d7acd72db360825371bb7977ff9679ffcc09dbf435cbaf2e3c32afb6bf7ad"}
Oct 30 19:19:37.118246 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:37.118203    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5" event={"ID":"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233","Type":"ContainerStarted","Data":"f1da6cce9f665398da85f3a603d674e9a647b6715e76b41ca0212055b43a1db8"}
Oct 30 19:19:37.118680 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:37.118386    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"
Oct 30 19:19:37.237235 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:37.237183    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5" podStartSLOduration=3.401046687 podStartE2EDuration="4.237169036s" totalImagesPullingTime="836.122349ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:37.235761209 +0000 UTC m=+2697.947509908" watchObservedRunningTime="2026-10-30 19:19:37.237169036 +0000 UTC m=+2697.948917733"
Oct 30 19:19:38.203602 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:19:38.203565    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:19:39.052980 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:39.046721    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:39.103887 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:39.103860    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"
Oct 30 19:19:39.237629 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:39.237593    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:43.129926 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:43.129890    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"
Oct 30 19:19:44.006962 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:44.006902    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:48.765782 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:48.765702    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:53.605436 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:53.605350    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:53.807235 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:53.807199    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:53.997297 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:53.997206    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:58.350374 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:58.350341    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"]
Oct 30 19:19:58.354904 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:58.354882    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"
Oct 30 19:19:58.409761 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:58.409723    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w0-19f6ea-serving-cert\""
Oct 30 19:19:58.415420 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:58.415389    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49ca5cb4-d828-417b-841f-c888ee4b5488-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d\" (UID: \"49ca5cb4-d828-417b-841f-c888ee4b5488\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"
Oct 30 19:19:58.415566 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:58.415437    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49ca5cb4-d828-417b-841f-c888ee4b5488-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d\" (UID: \"49ca5cb4-d828-417b-841f-c888ee4b5488\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"
Oct 30 19:19:58.415629 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:58.415568    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49ca5cb4-d828-417b-841f-c888ee4b5488-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d\" (UID: \"49ca5cb4-d828-417b-841f-c888ee4b5488\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"
Oct 30 19:19:58.415787 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:58.415764    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z7f7\" (UniqueName: \"kubernetes.io/projected/49ca5cb4-d828-417b-841f-c888ee4b5488-kube-api-access-9z7f7\") pod \"maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d\" (UID: \"49ca5cb4-d828-417b-841f-c888ee4b5488\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"
Oct 30 19:19:58.462233 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:58.462193    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"]
Oct 30 19:19:58.516774 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:58.516742    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z7f7\" (UniqueName: \"kubernetes.io/projected/49ca5cb4-d828-417b-841f-c888ee4b5488-kube-api-access-9z7f7\") pod \"maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d\" (UID: \"49ca5cb4-d828-417b-841f-c888ee4b5488\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"
Oct 30 19:19:58.516994 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:58.516915    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49ca5cb4-d828-417b-841f-c888ee4b5488-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d\" (UID: \"49ca5cb4-d828-417b-841f-c888ee4b5488\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"
Oct 30 19:19:58.516994 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:58.516978    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49ca5cb4-d828-417b-841f-c888ee4b5488-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d\" (UID: \"49ca5cb4-d828-417b-841f-c888ee4b5488\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"
Oct 30 19:19:58.517156 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:58.517035    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49ca5cb4-d828-417b-841f-c888ee4b5488-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d\" (UID: \"49ca5cb4-d828-417b-841f-c888ee4b5488\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"
Oct 30 19:19:58.517275 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:19:58.517255    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-serving-cert: secret "maas-api-e2e-shared-a-w0-19f6ea-serving-cert" not found
Oct 30 19:19:58.517357 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:19:58.517341    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49ca5cb4-d828-417b-841f-c888ee4b5488-maas-api-tls podName:49ca5cb4-d828-417b-841f-c888ee4b5488 nodeName:}" failed. No retries permitted until 2026-10-30 19:19:59.017320541 +0000 UTC m=+2719.729069230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/49ca5cb4-d828-417b-841f-c888ee4b5488-maas-api-tls") pod "maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d" (UID: "49ca5cb4-d828-417b-841f-c888ee4b5488") : secret "maas-api-e2e-shared-a-w0-19f6ea-serving-cert" not found
Oct 30 19:19:58.517771 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:58.517751    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49ca5cb4-d828-417b-841f-c888ee4b5488-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d\" (UID: \"49ca5cb4-d828-417b-841f-c888ee4b5488\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"
Oct 30 19:19:58.519848 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:58.519825    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49ca5cb4-d828-417b-841f-c888ee4b5488-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d\" (UID: \"49ca5cb4-d828-417b-841f-c888ee4b5488\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"
Oct 30 19:19:58.578462 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:58.578425    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z7f7\" (UniqueName: \"kubernetes.io/projected/49ca5cb4-d828-417b-841f-c888ee4b5488-kube-api-access-9z7f7\") pod \"maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d\" (UID: \"49ca5cb4-d828-417b-841f-c888ee4b5488\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"
Oct 30 19:19:58.744998 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:58.744865    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:58.929375 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:58.929336    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:59.040059 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:59.039921    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49ca5cb4-d828-417b-841f-c888ee4b5488-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d\" (UID: \"49ca5cb4-d828-417b-841f-c888ee4b5488\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"
Oct 30 19:19:59.042714 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:59.042684    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49ca5cb4-d828-417b-841f-c888ee4b5488-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d\" (UID: \"49ca5cb4-d828-417b-841f-c888ee4b5488\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"
Oct 30 19:19:59.282410 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:59.282367    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"
Oct 30 19:19:59.665613 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:19:59.665581    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"]
Oct 30 19:19:59.666624 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:19:59.666590    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49ca5cb4_d828_417b_841f_c888ee4b5488.slice/crio-bb7c24c785e03a6cbc57ce6c4df9d16f44805ef62528bf39bb0fdf66db1c26dd WatchSource:0}: Error finding container bb7c24c785e03a6cbc57ce6c4df9d16f44805ef62528bf39bb0fdf66db1c26dd: Status 404 returned error can't find the container with id bb7c24c785e03a6cbc57ce6c4df9d16f44805ef62528bf39bb0fdf66db1c26dd
Oct 30 19:20:00.307617 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:00.307586    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d" event={"ID":"49ca5cb4-d828-417b-841f-c888ee4b5488","Type":"ContainerStarted","Data":"bb7c24c785e03a6cbc57ce6c4df9d16f44805ef62528bf39bb0fdf66db1c26dd"}
Oct 30 19:20:01.318871 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:01.318822    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d" event={"ID":"49ca5cb4-d828-417b-841f-c888ee4b5488","Type":"ContainerStarted","Data":"2cebd08fa929f7a59cec6c01eeb5f9692fdf1c337ec3ed742bcab8c20e9207a1"}
Oct 30 19:20:01.319459 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:01.318989    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"
Oct 30 19:20:01.439618 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:01.439565    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d" podStartSLOduration=2.601706909 podStartE2EDuration="3.439546588s" totalImagesPullingTime="837.839679ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:01.437938954 +0000 UTC m=+2722.149687652" watchObservedRunningTime="2026-10-30 19:20:01.439546588 +0000 UTC m=+2722.151295285"
Oct 30 19:20:01.924767 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:01.924727    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:02.326908 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:02.326872    5820 generic.go:356] "Generic (PLEG): container finished" podID="795704e1-5f91-4636-a0b5-d9271dc4551d" containerID="02a2552ec01a773a0fedd7dc564aedc21963c9e9e654e0a47860038b1e93382e" exitCode=0
Oct 30 19:20:02.327323 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:02.326963    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2" event={"ID":"795704e1-5f91-4636-a0b5-d9271dc4551d","Type":"ContainerDied","Data":"02a2552ec01a773a0fedd7dc564aedc21963c9e9e654e0a47860038b1e93382e"}
Oct 30 19:20:02.327625 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:02.327609    5820 scope.go:122] "RemoveContainer" containerID="02a2552ec01a773a0fedd7dc564aedc21963c9e9e654e0a47860038b1e93382e"
Oct 30 19:20:02.329165 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:02.329138    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:20:02.329247 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:02.329221    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:20:02.329293 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:02.329250    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:20:03.357978 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:03.357438    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2" event={"ID":"795704e1-5f91-4636-a0b5-d9271dc4551d","Type":"ContainerStarted","Data":"3369c4799f7787fdff17f62a41cb6adfc659cc0180dde0c1278fd875ee84c46b"}
Oct 30 19:20:03.595662 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:03.595625    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:20:03.597177 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:03.597150    5820 patch_prober.go:29] interesting pod/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:20:03.597332 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:03.597207    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2" podUID="795704e1-5f91-4636-a0b5-d9271dc4551d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 30 19:20:03.668408 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:03.668311    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:20:03.790847 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:03.790752    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:04.596203 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:04.596167    5820 patch_prober.go:29] interesting pod/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:20:04.596641 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:04.596222    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2" podUID="795704e1-5f91-4636-a0b5-d9271dc4551d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 30 19:20:05.595428 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:05.595393    5820 patch_prober.go:29] interesting pod/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:20:05.595613 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:05.595446    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2" podUID="795704e1-5f91-4636-a0b5-d9271dc4551d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 30 19:20:06.595349 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:06.595309    5820 patch_prober.go:29] interesting pod/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:20:06.595789 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:06.595372    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2" podUID="795704e1-5f91-4636-a0b5-d9271dc4551d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 30 19:20:07.332789 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:07.332759    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"
Oct 30 19:20:07.595604 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:07.595517    5820 patch_prober.go:29] interesting pod/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:20:07.595604 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:07.595586    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2" podUID="795704e1-5f91-4636-a0b5-d9271dc4551d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 30 19:20:08.204341 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:20:08.204304    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:20:08.600318 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:08.600291    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:20:08.601496 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:08.601474    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:20:08.968837 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:08.968758    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:13.894615 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:13.894580    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:18.886111 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:18.885251    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:23.770983 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:23.769671    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:26.985877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:26.985830    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:27.213623 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:27.213586    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:28.595284 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:28.595252    5820 generic.go:356] "Generic (PLEG): container finished" podID="c1720171-99ee-4be0-8bbe-f8fdfed25319" containerID="8f6b68d55087a853db1204e07278fe0fa7cba762fed7fd73f9087270a41675d7" exitCode=0
Oct 30 19:20:28.595284 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:28.595295    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9" event={"ID":"c1720171-99ee-4be0-8bbe-f8fdfed25319","Type":"ContainerDied","Data":"8f6b68d55087a853db1204e07278fe0fa7cba762fed7fd73f9087270a41675d7"}
Oct 30 19:20:28.595748 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:28.595663    5820 scope.go:122] "RemoveContainer" containerID="8f6b68d55087a853db1204e07278fe0fa7cba762fed7fd73f9087270a41675d7"
Oct 30 19:20:28.597106 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:28.597076    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:20:28.597183 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:28.597146    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:20:28.597183 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:28.597179    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:20:28.647277 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:28.645366    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:29.602982 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:29.602929    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9" event={"ID":"c1720171-99ee-4be0-8bbe-f8fdfed25319","Type":"ContainerStarted","Data":"5bf0b2f26c81b72e3cf5b61d022c597172789cd89c88313676e66b74cccf58ea"}
Oct 30 19:20:30.206700 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:30.206657    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:20:30.208160 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:30.208134    5820 patch_prober.go:29] interesting pod/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 19:20:30.208294 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:30.208184    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9" podUID="c1720171-99ee-4be0-8bbe-f8fdfed25319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 19:20:30.854704 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:30.854672    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:20:31.207460 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:31.207380    5820 patch_prober.go:29] interesting pod/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 19:20:31.207460 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:31.207441    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9" podUID="c1720171-99ee-4be0-8bbe-f8fdfed25319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 19:20:32.207636 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:32.207599    5820 patch_prober.go:29] interesting pod/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 19:20:32.208157 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:32.207655    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9" podUID="c1720171-99ee-4be0-8bbe-f8fdfed25319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 19:20:33.207230 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:33.207194    5820 patch_prober.go:29] interesting pod/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 19:20:33.207404 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:33.207249    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9" podUID="c1720171-99ee-4be0-8bbe-f8fdfed25319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 19:20:33.714083 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:33.714044    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:34.207506 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:34.207471    5820 patch_prober.go:29] interesting pod/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 19:20:34.207692 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:34.207537    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9" podUID="c1720171-99ee-4be0-8bbe-f8fdfed25319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 19:20:35.207148 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:35.207116    5820 patch_prober.go:29] interesting pod/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 19:20:35.207620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:35.207174    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9" podUID="c1720171-99ee-4be0-8bbe-f8fdfed25319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 19:20:36.211902 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:36.211862    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:20:36.212751 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:36.212734    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:20:38.022802 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:38.022764    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:38.204631 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:20:38.204593    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:20:38.404762 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:38.404721    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:38.873565 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:38.873509    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:39.305512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:39.305211    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:39.563022 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:39.562916    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:43.275893 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:43.275861    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:43.680717 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:43.680529    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:44.274119 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:44.274079    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:44.470684 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:44.470651    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:44.739637 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:44.739600    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:45.006085 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:45.006000    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:45.216340 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:45.216299    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:45.942867 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:45.942777    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:47.171274 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:47.171215    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:47.730210 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:47.730110    5820 generic.go:356] "Generic (PLEG): container finished" podID="f4415a3e-5238-48e7-a6b2-e97c1904381f" containerID="d0721ee98511840ec85e512dbce762684b8399067cea19a5a2054635e6c3eb53" exitCode=0
Oct 30 19:20:47.730210 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:47.730160    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt" event={"ID":"f4415a3e-5238-48e7-a6b2-e97c1904381f","Type":"ContainerDied","Data":"d0721ee98511840ec85e512dbce762684b8399067cea19a5a2054635e6c3eb53"}
Oct 30 19:20:47.730601 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:47.730587    5820 scope.go:122] "RemoveContainer" containerID="d0721ee98511840ec85e512dbce762684b8399067cea19a5a2054635e6c3eb53"
Oct 30 19:20:47.732370 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:47.732339    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:20:47.732496 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:47.732420    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:20:47.732496 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:47.732456    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:20:48.272356 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:48.271641    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:48.500712 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:48.500675    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:48.668510 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:48.668467    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:20:48.748988 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:48.741100    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt" event={"ID":"f4415a3e-5238-48e7-a6b2-e97c1904381f","Type":"ContainerStarted","Data":"679e07438b23f5986fe508afd5d9ae4ae6f4d76a2273143e772749cb59fca21f"}
Oct 30 19:20:49.259006 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:49.257678    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:20:49.259006 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:49.257832    5820 patch_prober.go:29] interesting pod/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:20:49.259006 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:49.257893    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt" podUID="f4415a3e-5238-48e7-a6b2-e97c1904381f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused"
Oct 30 19:20:49.917978 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:49.917547    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:50.164979 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:50.163477    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:50.259021 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:50.255419    5820 patch_prober.go:29] interesting pod/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:20:50.259021 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:50.255523    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt" podUID="f4415a3e-5238-48e7-a6b2-e97c1904381f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused"
Oct 30 19:20:50.378333 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:50.378291    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:50.699285 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:50.699246    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:51.256978 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:51.254315    5820 patch_prober.go:29] interesting pod/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:20:51.256978 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:51.254378    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt" podUID="f4415a3e-5238-48e7-a6b2-e97c1904381f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused"
Oct 30 19:20:52.272975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:52.260160    5820 patch_prober.go:29] interesting pod/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:20:52.272975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:52.260224    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt" podUID="f4415a3e-5238-48e7-a6b2-e97c1904381f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused"
Oct 30 19:20:53.255976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:53.255290    5820 patch_prober.go:29] interesting pod/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:20:53.255976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:53.255355    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt" podUID="f4415a3e-5238-48e7-a6b2-e97c1904381f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused"
Oct 30 19:20:53.897535 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:53.897500    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:54.107459 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:54.107410    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:54.259268 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:54.259190    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:20:54.259604 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:54.259584    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:20:54.260255 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:54.260235    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:20:55.147292 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:55.147251    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:57.559914 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:57.559877    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:57.885028 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:57.881081    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:57.983758 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:57.983722    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd"]
Oct 30 19:20:57.986893 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:57.986876    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd"
Oct 30 19:20:58.051231 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:58.051203    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd"]
Oct 30 19:20:58.135968 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:58.135862    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-d4f84d\""
Oct 30 19:20:58.151405 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:58.151369    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-d4f84d-dockercfg-xxv6p\""
Oct 30 19:20:58.153678 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:58.153606    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snkrl\" (UniqueName: \"kubernetes.io/projected/4c05e9f8-0218-4456-8f76-1420649f533f-kube-api-access-snkrl\") pod \"payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd\" (UID: \"4c05e9f8-0218-4456-8f76-1420649f533f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd"
Oct 30 19:20:58.154153 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:58.154129    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c05e9f8-0218-4456-8f76-1420649f533f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd\" (UID: \"4c05e9f8-0218-4456-8f76-1420649f533f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd"
Oct 30 19:20:58.158656 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:58.158602    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh"]
Oct 30 19:20:58.202285 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:58.202251    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh"
Oct 30 19:20:58.202579 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:58.202275    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh"]
Oct 30 19:20:58.255270 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:58.255237    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c05e9f8-0218-4456-8f76-1420649f533f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd\" (UID: \"4c05e9f8-0218-4456-8f76-1420649f533f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd"
Oct 30 19:20:58.255395 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:58.255287    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snkrl\" (UniqueName: \"kubernetes.io/projected/4c05e9f8-0218-4456-8f76-1420649f533f-kube-api-access-snkrl\") pod \"payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd\" (UID: \"4c05e9f8-0218-4456-8f76-1420649f533f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd"
Oct 30 19:20:58.255395 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:58.255327    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54f97865-2f4e-4aa8-80d8-4e632aaf3847-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh\" (UID: \"54f97865-2f4e-4aa8-80d8-4e632aaf3847\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh"
Oct 30 19:20:58.255395 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:58.255351    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt5j5\" (UniqueName: \"kubernetes.io/projected/54f97865-2f4e-4aa8-80d8-4e632aaf3847-kube-api-access-zt5j5\") pod \"payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh\" (UID: \"54f97865-2f4e-4aa8-80d8-4e632aaf3847\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh"
Oct 30 19:20:58.256025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:58.256007    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c05e9f8-0218-4456-8f76-1420649f533f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd\" (UID: \"4c05e9f8-0218-4456-8f76-1420649f533f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd"
Oct 30 19:20:58.316139 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:58.316105    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snkrl\" (UniqueName: \"kubernetes.io/projected/4c05e9f8-0218-4456-8f76-1420649f533f-kube-api-access-snkrl\") pod \"payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd\" (UID: \"4c05e9f8-0218-4456-8f76-1420649f533f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd"
Oct 30 19:20:58.356618 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:58.356584    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54f97865-2f4e-4aa8-80d8-4e632aaf3847-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh\" (UID: \"54f97865-2f4e-4aa8-80d8-4e632aaf3847\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh"
Oct 30 19:20:58.356739 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:58.356635    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt5j5\" (UniqueName: \"kubernetes.io/projected/54f97865-2f4e-4aa8-80d8-4e632aaf3847-kube-api-access-zt5j5\") pod \"payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh\" (UID: \"54f97865-2f4e-4aa8-80d8-4e632aaf3847\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh"
Oct 30 19:20:58.357290 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:58.357269    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54f97865-2f4e-4aa8-80d8-4e632aaf3847-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh\" (UID: \"54f97865-2f4e-4aa8-80d8-4e632aaf3847\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh"
Oct 30 19:20:58.419648 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:58.419597    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt5j5\" (UniqueName: \"kubernetes.io/projected/54f97865-2f4e-4aa8-80d8-4e632aaf3847-kube-api-access-zt5j5\") pod \"payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh\" (UID: \"54f97865-2f4e-4aa8-80d8-4e632aaf3847\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh"
Oct 30 19:20:58.516752 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:58.516727    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh"
Oct 30 19:20:58.599498 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:58.599471    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd"
Oct 30 19:20:58.894894 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:58.894863    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh"]
Oct 30 19:20:58.896642 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:20:58.896618    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54f97865_2f4e_4aa8_80d8_4e632aaf3847.slice/crio-7894a9adbbbeebb7d6de34a85949cba243e091d9dc4a5d96f5424dcb4ac09faa WatchSource:0}: Error finding container 7894a9adbbbeebb7d6de34a85949cba243e091d9dc4a5d96f5424dcb4ac09faa: Status 404 returned error can't find the container with id 7894a9adbbbeebb7d6de34a85949cba243e091d9dc4a5d96f5424dcb4ac09faa
Oct 30 19:20:59.000965 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:59.000922    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd"]
Oct 30 19:20:59.001504 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:20:59.001480    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c05e9f8_0218_4456_8f76_1420649f533f.slice/crio-f6d205836c756e81c0bf1b42b32f077e60ce38aafad7cf54b716ed8fc562b213 WatchSource:0}: Error finding container f6d205836c756e81c0bf1b42b32f077e60ce38aafad7cf54b716ed8fc562b213: Status 404 returned error can't find the container with id f6d205836c756e81c0bf1b42b32f077e60ce38aafad7cf54b716ed8fc562b213
Oct 30 19:20:59.841980 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:59.841911    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd" event={"ID":"4c05e9f8-0218-4456-8f76-1420649f533f","Type":"ContainerStarted","Data":"fd313499defbd0309a13bfaebd633644b3420ffb57cd0b1612bd212a1c9a2308"}
Oct 30 19:20:59.842452 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:59.841991    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd" event={"ID":"4c05e9f8-0218-4456-8f76-1420649f533f","Type":"ContainerStarted","Data":"f6d205836c756e81c0bf1b42b32f077e60ce38aafad7cf54b716ed8fc562b213"}
Oct 30 19:20:59.842452 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:59.842022    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd"
Oct 30 19:20:59.843521 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:59.843493    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh" event={"ID":"54f97865-2f4e-4aa8-80d8-4e632aaf3847","Type":"ContainerStarted","Data":"92d1ebf2b3a591c4b56f8bf9ca928385413f4db45cd486a845bcc40628c4b8d7"}
Oct 30 19:20:59.843656 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:59.843536    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh" event={"ID":"54f97865-2f4e-4aa8-80d8-4e632aaf3847","Type":"ContainerStarted","Data":"7894a9adbbbeebb7d6de34a85949cba243e091d9dc4a5d96f5424dcb4ac09faa"}
Oct 30 19:20:59.843656 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:59.843566    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh"
Oct 30 19:20:59.961534 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:20:59.961487    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd" podStartSLOduration=2.961474752 podStartE2EDuration="2.961474752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:20:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:59.958398313 +0000 UTC m=+2780.670147012" watchObservedRunningTime="2026-10-30 19:20:59.961474752 +0000 UTC m=+2780.673223448"
Oct 30 19:21:00.315975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:00.314634    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:00.695247 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:00.695139    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:01.125454 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:01.125415    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:02.413629 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:02.413590    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:05.267131 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:05.267091    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:05.854346 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:05.846595    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:08.204414 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:21:08.204368    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:21:10.728520 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:10.728478    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:10.862999 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:10.862942    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh"
Oct 30 19:21:10.863177 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:10.863160    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd"
Oct 30 19:21:10.984001 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:10.983891    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh" podStartSLOduration=12.983876612 podStartE2EDuration="12.983876612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:20:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:00.092326264 +0000 UTC m=+2780.804074961" watchObservedRunningTime="2026-10-30 19:21:10.983876612 +0000 UTC m=+2791.695625309"
Oct 30 19:21:11.089689 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:11.089627    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:11.305241 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:11.305204    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:11.900909 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:11.900863    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:12.142661 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:12.142620    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:15.687725 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:15.687689    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:15.990405 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:15.990307    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:16.435720 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:16.435671    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:18.426338 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:18.426259    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z"]
Oct 30 19:21:18.523685 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:18.523652    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg"]
Oct 30 19:21:18.870971 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:18.866415    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:19.149390 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:19.149310    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:21.616398 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:21.616347    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:22.425434 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:22.425399    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:22.676608 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:22.676520    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:23.427241 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:23.427189    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z" podUID="e89efa10-f7ae-49b3-93b9-a939802f3f4e" containerName="payload-processing" containerID="cri-o://762eea0d0102a81b6f400d3de4a3c5894a5867c3ad7784e3865cf9862b370b8e" gracePeriod=25
Oct 30 19:21:23.524486 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:23.524451    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg" podUID="2f2b22b3-ad10-455d-a648-2100f39e3232" containerName="payload-pre-processing" containerID="cri-o://ecfe566adf1238afaca927ec6dadb6771e8c84e4f7788cb00d17c4c2d2d8bf37" gracePeriod=25
Oct 30 19:21:23.693630 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:23.693562    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z"
Oct 30 19:21:23.801070 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:23.801047    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg"
Oct 30 19:21:23.835704 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:23.835672    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e89efa10-f7ae-49b3-93b9-a939802f3f4e-kube-api-access-2tkbr\" (UniqueName: \"kubernetes.io/projected/e89efa10-f7ae-49b3-93b9-a939802f3f4e-kube-api-access-2tkbr\") pod \"e89efa10-f7ae-49b3-93b9-a939802f3f4e\" (UID: \"e89efa10-f7ae-49b3-93b9-a939802f3f4e\") "
Oct 30 19:21:23.835848 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:23.835803    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e89efa10-f7ae-49b3-93b9-a939802f3f4e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e89efa10-f7ae-49b3-93b9-a939802f3f4e-plugins-config-volume\") pod \"e89efa10-f7ae-49b3-93b9-a939802f3f4e\" (UID: \"e89efa10-f7ae-49b3-93b9-a939802f3f4e\") "
Oct 30 19:21:23.836160 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:23.836137    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e89efa10-f7ae-49b3-93b9-a939802f3f4e-plugins-config-volume" pod "e89efa10-f7ae-49b3-93b9-a939802f3f4e" (UID: "e89efa10-f7ae-49b3-93b9-a939802f3f4e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:21:23.836453 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:23.836428    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e89efa10-f7ae-49b3-93b9-a939802f3f4e-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:21:23.838431 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:23.838403    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e89efa10-f7ae-49b3-93b9-a939802f3f4e-kube-api-access-2tkbr" pod "e89efa10-f7ae-49b3-93b9-a939802f3f4e" (UID: "e89efa10-f7ae-49b3-93b9-a939802f3f4e"). InnerVolumeSpecName "kube-api-access-2tkbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:23.937411 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:23.937379    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f2b22b3-ad10-455d-a648-2100f39e3232-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f2b22b3-ad10-455d-a648-2100f39e3232-plugins-config-volume\") pod \"2f2b22b3-ad10-455d-a648-2100f39e3232\" (UID: \"2f2b22b3-ad10-455d-a648-2100f39e3232\") "
Oct 30 19:21:23.937587 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:23.937557    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f2b22b3-ad10-455d-a648-2100f39e3232-kube-api-access-gd7xj\" (UniqueName: \"kubernetes.io/projected/2f2b22b3-ad10-455d-a648-2100f39e3232-kube-api-access-gd7xj\") pod \"2f2b22b3-ad10-455d-a648-2100f39e3232\" (UID: \"2f2b22b3-ad10-455d-a648-2100f39e3232\") "
Oct 30 19:21:23.937780 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:23.937756    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f2b22b3-ad10-455d-a648-2100f39e3232-plugins-config-volume" pod "2f2b22b3-ad10-455d-a648-2100f39e3232" (UID: "2f2b22b3-ad10-455d-a648-2100f39e3232"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:21:23.938115 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:23.938091    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tkbr\" (UniqueName: \"kubernetes.io/projected/e89efa10-f7ae-49b3-93b9-a939802f3f4e-kube-api-access-2tkbr\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:21:23.938115 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:23.938117    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f2b22b3-ad10-455d-a648-2100f39e3232-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:21:23.940117 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:23.940091    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f2b22b3-ad10-455d-a648-2100f39e3232-kube-api-access-gd7xj" pod "2f2b22b3-ad10-455d-a648-2100f39e3232" (UID: "2f2b22b3-ad10-455d-a648-2100f39e3232"). InnerVolumeSpecName "kube-api-access-gd7xj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:24.038921 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:24.038835    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gd7xj\" (UniqueName: \"kubernetes.io/projected/2f2b22b3-ad10-455d-a648-2100f39e3232-kube-api-access-gd7xj\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:21:24.039081 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:24.039032    5820 generic.go:356] "Generic (PLEG): container finished" podID="e89efa10-f7ae-49b3-93b9-a939802f3f4e" containerID="762eea0d0102a81b6f400d3de4a3c5894a5867c3ad7784e3865cf9862b370b8e" exitCode=0
Oct 30 19:21:24.039148 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:24.039076    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z"
Oct 30 19:21:24.039148 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:24.039118    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z" event={"ID":"e89efa10-f7ae-49b3-93b9-a939802f3f4e","Type":"ContainerDied","Data":"762eea0d0102a81b6f400d3de4a3c5894a5867c3ad7784e3865cf9862b370b8e"}
Oct 30 19:21:24.039254 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:24.039184    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z" event={"ID":"e89efa10-f7ae-49b3-93b9-a939802f3f4e","Type":"ContainerDied","Data":"3b81823657ba83815c091a883e6144b3adafc3537e68ebaf51ecf92d1d5cb83b"}
Oct 30 19:21:24.039254 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:24.039226    5820 scope.go:122] "RemoveContainer" containerID="762eea0d0102a81b6f400d3de4a3c5894a5867c3ad7784e3865cf9862b370b8e"
Oct 30 19:21:24.040540 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:24.040516    5820 generic.go:356] "Generic (PLEG): container finished" podID="2f2b22b3-ad10-455d-a648-2100f39e3232" containerID="ecfe566adf1238afaca927ec6dadb6771e8c84e4f7788cb00d17c4c2d2d8bf37" exitCode=0
Oct 30 19:21:24.040643 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:24.040553    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg" event={"ID":"2f2b22b3-ad10-455d-a648-2100f39e3232","Type":"ContainerDied","Data":"ecfe566adf1238afaca927ec6dadb6771e8c84e4f7788cb00d17c4c2d2d8bf37"}
Oct 30 19:21:24.040643 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:24.040568    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg"
Oct 30 19:21:24.040643 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:24.040588    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg" event={"ID":"2f2b22b3-ad10-455d-a648-2100f39e3232","Type":"ContainerDied","Data":"a05fe0bb4c0eba6264eb0e0a903f1925853520ab7162cb5089c0b5a269eb0f9d"}
Oct 30 19:21:24.049634 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:24.049618    5820 scope.go:122] "RemoveContainer" containerID="762eea0d0102a81b6f400d3de4a3c5894a5867c3ad7784e3865cf9862b370b8e"
Oct 30 19:21:24.049976 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:21:24.049926    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"762eea0d0102a81b6f400d3de4a3c5894a5867c3ad7784e3865cf9862b370b8e\": container with ID starting with 762eea0d0102a81b6f400d3de4a3c5894a5867c3ad7784e3865cf9862b370b8e not found: ID does not exist" containerID="762eea0d0102a81b6f400d3de4a3c5894a5867c3ad7784e3865cf9862b370b8e"
Oct 30 19:21:24.050075 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:24.049971    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"762eea0d0102a81b6f400d3de4a3c5894a5867c3ad7784e3865cf9862b370b8e"} err="failed to get container status \"762eea0d0102a81b6f400d3de4a3c5894a5867c3ad7784e3865cf9862b370b8e\": rpc error: code = NotFound desc = could not find container \"762eea0d0102a81b6f400d3de4a3c5894a5867c3ad7784e3865cf9862b370b8e\": container with ID starting with 762eea0d0102a81b6f400d3de4a3c5894a5867c3ad7784e3865cf9862b370b8e not found: ID does not exist"
Oct 30 19:21:24.050075 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:24.049995    5820 scope.go:122] "RemoveContainer" containerID="ecfe566adf1238afaca927ec6dadb6771e8c84e4f7788cb00d17c4c2d2d8bf37"
Oct 30 19:21:24.059259 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:24.059241    5820 scope.go:122] "RemoveContainer" containerID="ecfe566adf1238afaca927ec6dadb6771e8c84e4f7788cb00d17c4c2d2d8bf37"
Oct 30 19:21:24.059517 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:21:24.059496    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecfe566adf1238afaca927ec6dadb6771e8c84e4f7788cb00d17c4c2d2d8bf37\": container with ID starting with ecfe566adf1238afaca927ec6dadb6771e8c84e4f7788cb00d17c4c2d2d8bf37 not found: ID does not exist" containerID="ecfe566adf1238afaca927ec6dadb6771e8c84e4f7788cb00d17c4c2d2d8bf37"
Oct 30 19:21:24.059567 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:24.059527    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecfe566adf1238afaca927ec6dadb6771e8c84e4f7788cb00d17c4c2d2d8bf37"} err="failed to get container status \"ecfe566adf1238afaca927ec6dadb6771e8c84e4f7788cb00d17c4c2d2d8bf37\": rpc error: code = NotFound desc = could not find container \"ecfe566adf1238afaca927ec6dadb6771e8c84e4f7788cb00d17c4c2d2d8bf37\": container with ID starting with ecfe566adf1238afaca927ec6dadb6771e8c84e4f7788cb00d17c4c2d2d8bf37 not found: ID does not exist"
Oct 30 19:21:24.204646 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:24.204616    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z"]
Oct 30 19:21:24.210714 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:24.210692    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-6d953c-6d8d8899b4-2lx5z"]
Oct 30 19:21:24.319647 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:24.319619    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg"]
Oct 30 19:21:24.324025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:24.323998    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-6d953c-78c767cdd7-fkwgg"]
Oct 30 19:21:25.438337 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:25.438308    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f2b22b3-ad10-455d-a648-2100f39e3232" path="/var/lib/kubelet/pods/2f2b22b3-ad10-455d-a648-2100f39e3232/volumes"
Oct 30 19:21:25.438720 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:25.438703    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e89efa10-f7ae-49b3-93b9-a939802f3f4e" path="/var/lib/kubelet/pods/e89efa10-f7ae-49b3-93b9-a939802f3f4e/volumes"
Oct 30 19:21:25.630876 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:25.630846    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"]
Oct 30 19:21:25.631074 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:25.631041    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97" podUID="3393cba7-4779-4143-b56e-879607b91b1b" containerName="maas-api" containerID="cri-o://29ee946faa835ac6865a72d4097f1cffca99d9b0335c5af792eb061769cda34a" gracePeriod=30
Oct 30 19:21:25.777140 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:25.777104    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:25.958281 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:25.958256    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"
Oct 30 19:21:26.032438 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:26.032337    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3393cba7-4779-4143-b56e-879607b91b1b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3393cba7-4779-4143-b56e-879607b91b1b-maas-api-tls\") pod \"3393cba7-4779-4143-b56e-879607b91b1b\" (UID: \"3393cba7-4779-4143-b56e-879607b91b1b\") "
Oct 30 19:21:26.032767 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:26.032746    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3393cba7-4779-4143-b56e-879607b91b1b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3393cba7-4779-4143-b56e-879607b91b1b-metrics-certs\") pod \"3393cba7-4779-4143-b56e-879607b91b1b\" (UID: \"3393cba7-4779-4143-b56e-879607b91b1b\") "
Oct 30 19:21:26.053213 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:26.053166    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3393cba7-4779-4143-b56e-879607b91b1b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3393cba7-4779-4143-b56e-879607b91b1b-openshift-service-ca\") pod \"3393cba7-4779-4143-b56e-879607b91b1b\" (UID: \"3393cba7-4779-4143-b56e-879607b91b1b\") "
Oct 30 19:21:26.062804 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:26.053968    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3393cba7-4779-4143-b56e-879607b91b1b-openshift-service-ca" pod "3393cba7-4779-4143-b56e-879607b91b1b" (UID: "3393cba7-4779-4143-b56e-879607b91b1b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:21:26.063016 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:26.062983    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3393cba7-4779-4143-b56e-879607b91b1b-maas-api-tls" pod "3393cba7-4779-4143-b56e-879607b91b1b" (UID: "3393cba7-4779-4143-b56e-879607b91b1b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:26.063573 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:26.063543    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3393cba7-4779-4143-b56e-879607b91b1b-metrics-certs" pod "3393cba7-4779-4143-b56e-879607b91b1b" (UID: "3393cba7-4779-4143-b56e-879607b91b1b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:26.066723 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:26.066697    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3393cba7-4779-4143-b56e-879607b91b1b-kube-api-access-h82jd\" (UniqueName: \"kubernetes.io/projected/3393cba7-4779-4143-b56e-879607b91b1b-kube-api-access-h82jd\") pod \"3393cba7-4779-4143-b56e-879607b91b1b\" (UID: \"3393cba7-4779-4143-b56e-879607b91b1b\") "
Oct 30 19:21:26.067775 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:26.067722    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3393cba7-4779-4143-b56e-879607b91b1b-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:21:26.067960 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:26.067931    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3393cba7-4779-4143-b56e-879607b91b1b-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:21:26.068154 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:26.068137    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3393cba7-4779-4143-b56e-879607b91b1b-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:21:26.070593 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:26.070566    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3393cba7-4779-4143-b56e-879607b91b1b-kube-api-access-h82jd" pod "3393cba7-4779-4143-b56e-879607b91b1b" (UID: "3393cba7-4779-4143-b56e-879607b91b1b"). InnerVolumeSpecName "kube-api-access-h82jd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:26.074842 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:26.074113    5820 generic.go:356] "Generic (PLEG): container finished" podID="3393cba7-4779-4143-b56e-879607b91b1b" containerID="29ee946faa835ac6865a72d4097f1cffca99d9b0335c5af792eb061769cda34a" exitCode=0
Oct 30 19:21:26.074842 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:26.074183    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"
Oct 30 19:21:26.074842 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:26.074396    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97" event={"ID":"3393cba7-4779-4143-b56e-879607b91b1b","Type":"ContainerDied","Data":"29ee946faa835ac6865a72d4097f1cffca99d9b0335c5af792eb061769cda34a"}
Oct 30 19:21:26.074842 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:26.074502    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97" event={"ID":"3393cba7-4779-4143-b56e-879607b91b1b","Type":"ContainerDied","Data":"fb91df8b6c45d43c36a972c737647a4137a127ecdd4dc703a5cd9cd7c3d6697c"}
Oct 30 19:21:26.074842 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:26.074546    5820 scope.go:122] "RemoveContainer" containerID="29ee946faa835ac6865a72d4097f1cffca99d9b0335c5af792eb061769cda34a"
Oct 30 19:21:26.085700 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:26.085681    5820 scope.go:122] "RemoveContainer" containerID="29ee946faa835ac6865a72d4097f1cffca99d9b0335c5af792eb061769cda34a"
Oct 30 19:21:26.086036 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:21:26.086014    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29ee946faa835ac6865a72d4097f1cffca99d9b0335c5af792eb061769cda34a\": container with ID starting with 29ee946faa835ac6865a72d4097f1cffca99d9b0335c5af792eb061769cda34a not found: ID does not exist" containerID="29ee946faa835ac6865a72d4097f1cffca99d9b0335c5af792eb061769cda34a"
Oct 30 19:21:26.086123 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:26.086050    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29ee946faa835ac6865a72d4097f1cffca99d9b0335c5af792eb061769cda34a"} err="failed to get container status \"29ee946faa835ac6865a72d4097f1cffca99d9b0335c5af792eb061769cda34a\": rpc error: code = NotFound desc = could not find container \"29ee946faa835ac6865a72d4097f1cffca99d9b0335c5af792eb061769cda34a\": container with ID starting with 29ee946faa835ac6865a72d4097f1cffca99d9b0335c5af792eb061769cda34a not found: ID does not exist"
Oct 30 19:21:26.169624 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:26.169595    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h82jd\" (UniqueName: \"kubernetes.io/projected/3393cba7-4779-4143-b56e-879607b91b1b-kube-api-access-h82jd\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:21:26.211695 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:26.211653    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"]
Oct 30 19:21:26.219813 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:26.219770    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-6d953c-6c97f469b8-2rp97"]
Oct 30 19:21:27.305695 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:27.305661    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:27.437782 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:27.437750    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3393cba7-4779-4143-b56e-879607b91b1b" path="/var/lib/kubelet/pods/3393cba7-4779-4143-b56e-879607b91b1b/volumes"
Oct 30 19:21:27.535367 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:27.535138    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:27.725315 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:27.725225    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:28.251166 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:28.251133    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:29.384014 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:29.383976    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:32.785110 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:32.785069    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:34.987179 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:34.987144    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:35.224354 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:35.224319    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:35.426499 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:35.426470    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:38.031971 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:38.028879    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:38.204997 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:21:38.204929    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:21:42.650592 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.650552    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"]
Oct 30 19:21:42.651442 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.651412    5820 cpu_manager.go:414] "containerMap: removing container" podUID="e89efa10-f7ae-49b3-93b9-a939802f3f4e" containerName="payload-processing"
Oct 30 19:21:42.651442 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.651435    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e89efa10-f7ae-49b3-93b9-a939802f3f4e" containerName="payload-processing"
Oct 30 19:21:42.651605 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.651460    5820 cpu_manager.go:414] "containerMap: removing container" podUID="3393cba7-4779-4143-b56e-879607b91b1b" containerName="maas-api"
Oct 30 19:21:42.651605 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.651466    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3393cba7-4779-4143-b56e-879607b91b1b" containerName="maas-api"
Oct 30 19:21:42.651605 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.651483    5820 cpu_manager.go:414] "containerMap: removing container" podUID="2f2b22b3-ad10-455d-a648-2100f39e3232" containerName="payload-pre-processing"
Oct 30 19:21:42.651605 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.651488    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f2b22b3-ad10-455d-a648-2100f39e3232" containerName="payload-pre-processing"
Oct 30 19:21:42.651605 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.651593    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="e89efa10-f7ae-49b3-93b9-a939802f3f4e" containerName="payload-processing"
Oct 30 19:21:42.651605 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.651604    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="3393cba7-4779-4143-b56e-879607b91b1b" containerName="maas-api"
Oct 30 19:21:42.651801 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.651613    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f2b22b3-ad10-455d-a648-2100f39e3232" containerName="payload-pre-processing"
Oct 30 19:21:42.656293 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.656276    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"
Oct 30 19:21:42.712227 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.712195    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-dc5cdd-serving-cert\""
Oct 30 19:21:42.719844 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.719818    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"]
Oct 30 19:21:42.720199 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.720177    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85a27989-15d4-4c57-bdcd-1ce98d0811b2-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq\" (UID: \"85a27989-15d4-4c57-bdcd-1ce98d0811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"
Oct 30 19:21:42.720255 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.720240    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85a27989-15d4-4c57-bdcd-1ce98d0811b2-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq\" (UID: \"85a27989-15d4-4c57-bdcd-1ce98d0811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"
Oct 30 19:21:42.720383 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.720360    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzg9c\" (UniqueName: \"kubernetes.io/projected/85a27989-15d4-4c57-bdcd-1ce98d0811b2-kube-api-access-gzg9c\") pod \"maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq\" (UID: \"85a27989-15d4-4c57-bdcd-1ce98d0811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"
Oct 30 19:21:42.720423 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.720404    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85a27989-15d4-4c57-bdcd-1ce98d0811b2-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq\" (UID: \"85a27989-15d4-4c57-bdcd-1ce98d0811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"
Oct 30 19:21:42.821593 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.821560    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85a27989-15d4-4c57-bdcd-1ce98d0811b2-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq\" (UID: \"85a27989-15d4-4c57-bdcd-1ce98d0811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"
Oct 30 19:21:42.821780 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.821647    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzg9c\" (UniqueName: \"kubernetes.io/projected/85a27989-15d4-4c57-bdcd-1ce98d0811b2-kube-api-access-gzg9c\") pod \"maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq\" (UID: \"85a27989-15d4-4c57-bdcd-1ce98d0811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"
Oct 30 19:21:42.821780 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:21:42.821733    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-serving-cert: secret "maas-api-e2e-shared-b-w2-dc5cdd-serving-cert" not found
Oct 30 19:21:42.821881 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.821792    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85a27989-15d4-4c57-bdcd-1ce98d0811b2-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq\" (UID: \"85a27989-15d4-4c57-bdcd-1ce98d0811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"
Oct 30 19:21:42.821881 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:21:42.821808    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85a27989-15d4-4c57-bdcd-1ce98d0811b2-maas-api-tls podName:85a27989-15d4-4c57-bdcd-1ce98d0811b2 nodeName:}" failed. No retries permitted until 2026-10-30 19:21:43.321789484 +0000 UTC m=+2824.033538160 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/85a27989-15d4-4c57-bdcd-1ce98d0811b2-maas-api-tls") pod "maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq" (UID: "85a27989-15d4-4c57-bdcd-1ce98d0811b2") : secret "maas-api-e2e-shared-b-w2-dc5cdd-serving-cert" not found
Oct 30 19:21:42.822027 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.821987    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85a27989-15d4-4c57-bdcd-1ce98d0811b2-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq\" (UID: \"85a27989-15d4-4c57-bdcd-1ce98d0811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"
Oct 30 19:21:42.822548 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.822530    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85a27989-15d4-4c57-bdcd-1ce98d0811b2-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq\" (UID: \"85a27989-15d4-4c57-bdcd-1ce98d0811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"
Oct 30 19:21:42.824390 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.824353    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85a27989-15d4-4c57-bdcd-1ce98d0811b2-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq\" (UID: \"85a27989-15d4-4c57-bdcd-1ce98d0811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"
Oct 30 19:21:42.883203 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.883172    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzg9c\" (UniqueName: \"kubernetes.io/projected/85a27989-15d4-4c57-bdcd-1ce98d0811b2-kube-api-access-gzg9c\") pod \"maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq\" (UID: \"85a27989-15d4-4c57-bdcd-1ce98d0811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"
Oct 30 19:21:42.885541 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:42.885510    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:43.119787 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:43.119747    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:43.315551 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:43.315517    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:43.331395 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:43.331365    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85a27989-15d4-4c57-bdcd-1ce98d0811b2-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq\" (UID: \"85a27989-15d4-4c57-bdcd-1ce98d0811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"
Oct 30 19:21:43.333944 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:43.333928    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85a27989-15d4-4c57-bdcd-1ce98d0811b2-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq\" (UID: \"85a27989-15d4-4c57-bdcd-1ce98d0811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"
Oct 30 19:21:43.570320 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:43.570279    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"
Oct 30 19:21:43.953387 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:43.953351    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"]
Oct 30 19:21:43.954775 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:21:43.954746    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85a27989_15d4_4c57_bdcd_1ce98d0811b2.slice/crio-a67bdfa1bb8d387825ec94555907ce35f6145942e71e910f5925253e498abed0 WatchSource:0}: Error finding container a67bdfa1bb8d387825ec94555907ce35f6145942e71e910f5925253e498abed0: Status 404 returned error can't find the container with id a67bdfa1bb8d387825ec94555907ce35f6145942e71e910f5925253e498abed0
Oct 30 19:21:44.194372 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:44.194343    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq" event={"ID":"85a27989-15d4-4c57-bdcd-1ce98d0811b2","Type":"ContainerStarted","Data":"a67bdfa1bb8d387825ec94555907ce35f6145942e71e910f5925253e498abed0"}
Oct 30 19:21:45.201666 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:45.201580    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq" event={"ID":"85a27989-15d4-4c57-bdcd-1ce98d0811b2","Type":"ContainerStarted","Data":"e4d73e22f3fd18d7c2524d1cd092d946f8752425126bf5f002f6e3b318fa4aa2"}
Oct 30 19:21:45.202052 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:45.201665    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"
Oct 30 19:21:45.321011 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:45.320934    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq" podStartSLOduration=2.429740613 podStartE2EDuration="3.320919928s" totalImagesPullingTime="891.179315ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:45.318588531 +0000 UTC m=+2826.030337228" watchObservedRunningTime="2026-10-30 19:21:45.320919928 +0000 UTC m=+2826.032668624"
Oct 30 19:21:47.869941 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:47.868218    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:50.114552 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:50.114517    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp"]
Oct 30 19:21:50.554969 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:50.554914    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:50.986712 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:50.986619    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:51.213268 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:51.213235    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"
Oct 30 19:21:52.025727 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:52.025692    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:52.311596 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:52.311557    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:52.434556 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:52.434531    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"]
Oct 30 19:21:52.438873 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:52.438857    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"
Oct 30 19:21:52.491018 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:52.490994    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-d4f84d-serving-cert\""
Oct 30 19:21:52.544229 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:52.544195    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pl4j\" (UniqueName: \"kubernetes.io/projected/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-kube-api-access-9pl4j\") pod \"maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg\" (UID: \"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"
Oct 30 19:21:52.544420 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:52.544326    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg\" (UID: \"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"
Oct 30 19:21:52.544497 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:52.544439    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg\" (UID: \"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"
Oct 30 19:21:52.545105 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:52.544726    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg\" (UID: \"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"
Oct 30 19:21:52.545332 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:52.545313    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"]
Oct 30 19:21:52.646361 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:52.646281    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg\" (UID: \"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"
Oct 30 19:21:52.646501 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:52.646385    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pl4j\" (UniqueName: \"kubernetes.io/projected/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-kube-api-access-9pl4j\") pod \"maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg\" (UID: \"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"
Oct 30 19:21:52.646501 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:52.646417    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg\" (UID: \"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"
Oct 30 19:21:52.646501 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:52.646440    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg\" (UID: \"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"
Oct 30 19:21:52.646673 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:21:52.646536    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-serving-cert: secret "maas-api-e2e-shared-b-w1-d4f84d-serving-cert" not found
Oct 30 19:21:52.646673 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:21:52.646620    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-maas-api-tls podName:9e354da3-bcc8-41f9-ad39-ffdf8b9596c6 nodeName:}" failed. No retries permitted until 2026-10-30 19:21:53.146598464 +0000 UTC m=+2833.858347158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-maas-api-tls") pod "maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg" (UID: "9e354da3-bcc8-41f9-ad39-ffdf8b9596c6") : secret "maas-api-e2e-shared-b-w1-d4f84d-serving-cert" not found
Oct 30 19:21:52.647180 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:52.647156    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg\" (UID: \"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"
Oct 30 19:21:52.649007 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:52.648990    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg\" (UID: \"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"
Oct 30 19:21:52.717402 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:52.717363    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pl4j\" (UniqueName: \"kubernetes.io/projected/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-kube-api-access-9pl4j\") pod \"maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg\" (UID: \"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"
Oct 30 19:21:52.729550 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:52.729477    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:52.988904 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:52.988813    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:53.170942 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:53.170909    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg\" (UID: \"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"
Oct 30 19:21:53.173541 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:53.173520    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg\" (UID: \"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"
Oct 30 19:21:53.354242 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:53.354206    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"
Oct 30 19:21:53.732031 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:53.732009    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"]
Oct 30 19:21:53.734112 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:21:53.734086    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e354da3_bcc8_41f9_ad39_ffdf8b9596c6.slice/crio-319303f4e19877510dc23a2559c4881b370c88b4e32ecf7668f2cdf9636fa191 WatchSource:0}: Error finding container 319303f4e19877510dc23a2559c4881b370c88b4e32ecf7668f2cdf9636fa191: Status 404 returned error can't find the container with id 319303f4e19877510dc23a2559c4881b370c88b4e32ecf7668f2cdf9636fa191
Oct 30 19:21:54.259255 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:54.259222    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg" event={"ID":"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6","Type":"ContainerStarted","Data":"319303f4e19877510dc23a2559c4881b370c88b4e32ecf7668f2cdf9636fa191"}
Oct 30 19:21:55.114859 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:55.114817    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp" podUID="88a930a8-b6ff-4b13-a9d0-6e4728e55360" containerName="payload-processing" containerID="cri-o://f2203a5ef3a85f9c4e0cc99806f4eaad3c1de6a132e9744284a442303f708008" gracePeriod=25
Oct 30 19:21:55.270900 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:55.270865    5820 generic.go:356] "Generic (PLEG): container finished" podID="88a930a8-b6ff-4b13-a9d0-6e4728e55360" containerID="f2203a5ef3a85f9c4e0cc99806f4eaad3c1de6a132e9744284a442303f708008" exitCode=0
Oct 30 19:21:55.271092 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:55.270941    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp" event={"ID":"88a930a8-b6ff-4b13-a9d0-6e4728e55360","Type":"ContainerDied","Data":"f2203a5ef3a85f9c4e0cc99806f4eaad3c1de6a132e9744284a442303f708008"}
Oct 30 19:21:55.273161 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:55.273129    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg" event={"ID":"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6","Type":"ContainerStarted","Data":"10ca758782f0f624b096ba8fc540107952e5ad9ad4f2e731da13c66be6841c41"}
Oct 30 19:21:55.273311 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:55.273251    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"
Oct 30 19:21:55.393886 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:55.393831    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg" podStartSLOduration=2.461868047 podStartE2EDuration="3.393816525s" totalImagesPullingTime="931.948478ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:55.391249317 +0000 UTC m=+2836.102998013" watchObservedRunningTime="2026-10-30 19:21:55.393816525 +0000 UTC m=+2836.105565261"
Oct 30 19:21:55.410810 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:55.410783    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp"
Oct 30 19:21:55.501899 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:55.501871    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/88a930a8-b6ff-4b13-a9d0-6e4728e55360-kube-api-access-ssznw\" (UniqueName: \"kubernetes.io/projected/88a930a8-b6ff-4b13-a9d0-6e4728e55360-kube-api-access-ssznw\") pod \"88a930a8-b6ff-4b13-a9d0-6e4728e55360\" (UID: \"88a930a8-b6ff-4b13-a9d0-6e4728e55360\") "
Oct 30 19:21:55.502077 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:55.502028    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/88a930a8-b6ff-4b13-a9d0-6e4728e55360-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88a930a8-b6ff-4b13-a9d0-6e4728e55360-plugins-config-volume\") pod \"88a930a8-b6ff-4b13-a9d0-6e4728e55360\" (UID: \"88a930a8-b6ff-4b13-a9d0-6e4728e55360\") "
Oct 30 19:21:55.502393 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:55.502363    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88a930a8-b6ff-4b13-a9d0-6e4728e55360-plugins-config-volume" pod "88a930a8-b6ff-4b13-a9d0-6e4728e55360" (UID: "88a930a8-b6ff-4b13-a9d0-6e4728e55360"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:21:55.502540 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:55.502524    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88a930a8-b6ff-4b13-a9d0-6e4728e55360-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:21:55.504098 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:55.504061    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88a930a8-b6ff-4b13-a9d0-6e4728e55360-kube-api-access-ssznw" pod "88a930a8-b6ff-4b13-a9d0-6e4728e55360" (UID: "88a930a8-b6ff-4b13-a9d0-6e4728e55360"). InnerVolumeSpecName "kube-api-access-ssznw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:55.603457 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:55.603423    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssznw\" (UniqueName: \"kubernetes.io/projected/88a930a8-b6ff-4b13-a9d0-6e4728e55360-kube-api-access-ssznw\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:21:56.280082 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:56.280047    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp"
Oct 30 19:21:56.280082 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:56.280076    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp" event={"ID":"88a930a8-b6ff-4b13-a9d0-6e4728e55360","Type":"ContainerDied","Data":"1a7d94fc6320af4f14739d52ad17f256ec4196db00cfe453542d129026941592"}
Oct 30 19:21:56.280602 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:56.280130    5820 scope.go:122] "RemoveContainer" containerID="f2203a5ef3a85f9c4e0cc99806f4eaad3c1de6a132e9744284a442303f708008"
Oct 30 19:21:56.446144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:56.446116    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp"]
Oct 30 19:21:56.450015 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:56.449988    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-9f939e-79597446d4-z5snp"]
Oct 30 19:21:57.438062 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:57.438026    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="88a930a8-b6ff-4b13-a9d0-6e4728e55360" path="/var/lib/kubelet/pods/88a930a8-b6ff-4b13-a9d0-6e4728e55360/volumes"
Oct 30 19:21:57.695669 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:21:57.695587    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:01.285252 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:01.285215    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"
Oct 30 19:22:02.240292 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.240219    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"]
Oct 30 19:22:02.240861 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.240846    5820 cpu_manager.go:414] "containerMap: removing container" podUID="88a930a8-b6ff-4b13-a9d0-6e4728e55360" containerName="payload-processing"
Oct 30 19:22:02.240906 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.240863    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="88a930a8-b6ff-4b13-a9d0-6e4728e55360" containerName="payload-processing"
Oct 30 19:22:02.241053 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.241040    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="88a930a8-b6ff-4b13-a9d0-6e4728e55360" containerName="payload-processing"
Oct 30 19:22:02.246060 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.246029    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"
Oct 30 19:22:02.298232 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.298209    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-f5dfb8-serving-cert\""
Oct 30 19:22:02.306841 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.306819    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"]
Oct 30 19:22:02.378926 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.378893    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28cp8\" (UniqueName: \"kubernetes.io/projected/a84845b1-4310-4b17-949d-e4ee705f240e-kube-api-access-28cp8\") pod \"maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp\" (UID: \"a84845b1-4310-4b17-949d-e4ee705f240e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"
Oct 30 19:22:02.379143 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.379112    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a84845b1-4310-4b17-949d-e4ee705f240e-metrics-certs\") pod \"maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp\" (UID: \"a84845b1-4310-4b17-949d-e4ee705f240e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"
Oct 30 19:22:02.379213 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.379188    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a84845b1-4310-4b17-949d-e4ee705f240e-maas-api-tls\") pod \"maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp\" (UID: \"a84845b1-4310-4b17-949d-e4ee705f240e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"
Oct 30 19:22:02.379297 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.379278    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a84845b1-4310-4b17-949d-e4ee705f240e-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp\" (UID: \"a84845b1-4310-4b17-949d-e4ee705f240e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"
Oct 30 19:22:02.480734 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.480700    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28cp8\" (UniqueName: \"kubernetes.io/projected/a84845b1-4310-4b17-949d-e4ee705f240e-kube-api-access-28cp8\") pod \"maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp\" (UID: \"a84845b1-4310-4b17-949d-e4ee705f240e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"
Oct 30 19:22:02.480980 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.480830    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a84845b1-4310-4b17-949d-e4ee705f240e-metrics-certs\") pod \"maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp\" (UID: \"a84845b1-4310-4b17-949d-e4ee705f240e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"
Oct 30 19:22:02.480980 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.480893    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a84845b1-4310-4b17-949d-e4ee705f240e-maas-api-tls\") pod \"maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp\" (UID: \"a84845b1-4310-4b17-949d-e4ee705f240e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"
Oct 30 19:22:02.481118 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.481007    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a84845b1-4310-4b17-949d-e4ee705f240e-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp\" (UID: \"a84845b1-4310-4b17-949d-e4ee705f240e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"
Oct 30 19:22:02.482595 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.481796    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a84845b1-4310-4b17-949d-e4ee705f240e-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp\" (UID: \"a84845b1-4310-4b17-949d-e4ee705f240e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"
Oct 30 19:22:02.482595 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:22:02.481901    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-serving-cert: secret "maas-api-e2e-praxis-b-f5dfb8-serving-cert" not found
Oct 30 19:22:02.482595 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:22:02.481995    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a84845b1-4310-4b17-949d-e4ee705f240e-maas-api-tls podName:a84845b1-4310-4b17-949d-e4ee705f240e nodeName:}" failed. No retries permitted until 2026-10-30 19:22:02.981974865 +0000 UTC m=+2843.693723557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a84845b1-4310-4b17-949d-e4ee705f240e-maas-api-tls") pod "maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp" (UID: "a84845b1-4310-4b17-949d-e4ee705f240e") : secret "maas-api-e2e-praxis-b-f5dfb8-serving-cert" not found
Oct 30 19:22:02.510651 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.510551    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:02.535103 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.532819    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a84845b1-4310-4b17-949d-e4ee705f240e-metrics-certs\") pod \"maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp\" (UID: \"a84845b1-4310-4b17-949d-e4ee705f240e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"
Oct 30 19:22:02.544117 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.544083    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28cp8\" (UniqueName: \"kubernetes.io/projected/a84845b1-4310-4b17-949d-e4ee705f240e-kube-api-access-28cp8\") pod \"maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp\" (UID: \"a84845b1-4310-4b17-949d-e4ee705f240e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"
Oct 30 19:22:02.710046 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.710002    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:02.918562 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.918520    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:02.986580 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.986545    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a84845b1-4310-4b17-949d-e4ee705f240e-maas-api-tls\") pod \"maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp\" (UID: \"a84845b1-4310-4b17-949d-e4ee705f240e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"
Oct 30 19:22:02.989138 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:02.989117    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a84845b1-4310-4b17-949d-e4ee705f240e-maas-api-tls\") pod \"maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp\" (UID: \"a84845b1-4310-4b17-949d-e4ee705f240e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"
Oct 30 19:22:03.159788 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:03.159756    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"
Oct 30 19:22:03.568801 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:03.568777    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"]
Oct 30 19:22:03.570735 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:22:03.570708    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda84845b1_4310_4b17_949d_e4ee705f240e.slice/crio-581ce0e7fb5bb4adf92b0acf45bc2d39fe66df06bd8ee7436f7c5df22fbf3a19 WatchSource:0}: Error finding container 581ce0e7fb5bb4adf92b0acf45bc2d39fe66df06bd8ee7436f7c5df22fbf3a19: Status 404 returned error can't find the container with id 581ce0e7fb5bb4adf92b0acf45bc2d39fe66df06bd8ee7436f7c5df22fbf3a19
Oct 30 19:22:04.333629 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:04.333597    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp" event={"ID":"a84845b1-4310-4b17-949d-e4ee705f240e","Type":"ContainerStarted","Data":"581ce0e7fb5bb4adf92b0acf45bc2d39fe66df06bd8ee7436f7c5df22fbf3a19"}
Oct 30 19:22:05.341211 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:05.341175    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp" event={"ID":"a84845b1-4310-4b17-949d-e4ee705f240e","Type":"ContainerStarted","Data":"fd5e152102f198484d7bb011318c4dfe9af393e62bb89fbb766a52a567282ffe"}
Oct 30 19:22:05.341699 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:05.341237    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"
Oct 30 19:22:05.461595 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:05.461540    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp" podStartSLOduration=2.630530777 podStartE2EDuration="3.461526637s" totalImagesPullingTime="830.99586ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:05.459181532 +0000 UTC m=+2846.170930231" watchObservedRunningTime="2026-10-30 19:22:05.461526637 +0000 UTC m=+2846.173275333"
Oct 30 19:22:06.839814 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:06.839778    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:08.203333 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:22:08.203293    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:22:08.359642 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:08.359605    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:08.632799 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:08.632761    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:08.871091 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:08.871052    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:09.337271 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:09.337238    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:11.354699 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:11.354673    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"
Oct 30 19:22:13.803127 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:13.803098    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:16.785307 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:16.785269    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:17.085262 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:17.085221    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:17.339659 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:17.339554    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:18.625543 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:18.625507    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:23.422443 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:23.422392    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:23.697902 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:23.697803    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:23.890287 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:23.890257    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:28.445170 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:28.445129    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:33.292184 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:33.292144    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:34.065652 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:34.065614    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:35.063262 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:35.061421    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:35.330163 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:35.330078    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:38.187610 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:38.187576    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:38.211194 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:22:38.211159    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:22:39.733318 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:39.733279    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:39.968285 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:39.968241    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:43.043061 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:43.043025    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:43.253183 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:43.253137    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:43.800832 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:43.800797    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:49.136982 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:49.136859    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:54.002171 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:54.002121    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:56.019019 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:56.018978    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:59.043915 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:22:59.043875    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:02.172030 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:02.171596    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:02.932575 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:02.932540    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:04.526982 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:04.524483    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:06.148060 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:06.148025    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:08.203347 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:23:08.203313    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:23:09.758892 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:09.757036    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:12.136242 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:12.136089    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:13.477187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:13.477153    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"]
Oct 30 19:23:13.477747 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:13.477361    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm" podUID="2c64155a-21e5-4b7c-99f9-f2d35fc457b0" containerName="istio-proxy" containerID="cri-o://56344f47feeeb1209c55b56f1b0503ab319b3eb7c0053019230e6fad150e1841" gracePeriod=30
Oct 30 19:23:15.414453 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:15.414416    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:18.737740 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.737703    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:23:18.833921 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.833890    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-kube-api-access-sq8bv\" (UniqueName: \"kubernetes.io/projected/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-kube-api-access-sq8bv\") pod \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") "
Oct 30 19:23:18.834143 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.833988    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-workload-socket\") pod \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") "
Oct 30 19:23:18.834143 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.834044    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-workload-certs\") pod \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") "
Oct 30 19:23:18.834143 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.834074    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istiod-ca-cert\") pod \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") "
Oct 30 19:23:18.834143 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.834133    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-data\") pod \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") "
Oct 30 19:23:18.834339 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.834184    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-envoy\") pod \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") "
Oct 30 19:23:18.834339 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.834225    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-credential-socket\") pod \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") "
Oct 30 19:23:18.834339 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.834295    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-token\" (UniqueName: \"kubernetes.io/projected/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-token\") pod \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") "
Oct 30 19:23:18.834509 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.834356    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-workload-certs" pod "2c64155a-21e5-4b7c-99f9-f2d35fc457b0" (UID: "2c64155a-21e5-4b7c-99f9-f2d35fc457b0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:18.834626 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.834607    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-workload-socket" pod "2c64155a-21e5-4b7c-99f9-f2d35fc457b0" (UID: "2c64155a-21e5-4b7c-99f9-f2d35fc457b0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:18.834700 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.834644    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-data" pod "2c64155a-21e5-4b7c-99f9-f2d35fc457b0" (UID: "2c64155a-21e5-4b7c-99f9-f2d35fc457b0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:18.834913 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.834894    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:23:18.834913 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.834903    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istiod-ca-cert" pod "2c64155a-21e5-4b7c-99f9-f2d35fc457b0" (UID: "2c64155a-21e5-4b7c-99f9-f2d35fc457b0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:23:18.835051 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.834914    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:23:18.835051 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.834929    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:23:18.835137 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.835091    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-credential-socket" pod "2c64155a-21e5-4b7c-99f9-f2d35fc457b0" (UID: "2c64155a-21e5-4b7c-99f9-f2d35fc457b0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:18.836662 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.836636    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-envoy" pod "2c64155a-21e5-4b7c-99f9-f2d35fc457b0" (UID: "2c64155a-21e5-4b7c-99f9-f2d35fc457b0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:18.836864 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.836846    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-kube-api-access-sq8bv" pod "2c64155a-21e5-4b7c-99f9-f2d35fc457b0" (UID: "2c64155a-21e5-4b7c-99f9-f2d35fc457b0"). InnerVolumeSpecName "kube-api-access-sq8bv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:23:18.836967 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.836932    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-token" pod "2c64155a-21e5-4b7c-99f9-f2d35fc457b0" (UID: "2c64155a-21e5-4b7c-99f9-f2d35fc457b0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:23:18.842767 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.842742    5820 generic.go:356] "Generic (PLEG): container finished" podID="2c64155a-21e5-4b7c-99f9-f2d35fc457b0" containerID="56344f47feeeb1209c55b56f1b0503ab319b3eb7c0053019230e6fad150e1841" exitCode=0
Oct 30 19:23:18.842889 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.842794    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"
Oct 30 19:23:18.842889 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.842821    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm" event={"ID":"2c64155a-21e5-4b7c-99f9-f2d35fc457b0","Type":"ContainerDied","Data":"56344f47feeeb1209c55b56f1b0503ab319b3eb7c0053019230e6fad150e1841"}
Oct 30 19:23:18.842889 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.842870    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm" event={"ID":"2c64155a-21e5-4b7c-99f9-f2d35fc457b0","Type":"ContainerDied","Data":"a972e9f6c8429f29bba7259bf0c2647e811f97dd52923dec7e1c43273ea699c2"}
Oct 30 19:23:18.843087 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.842893    5820 scope.go:122] "RemoveContainer" containerID="56344f47feeeb1209c55b56f1b0503ab319b3eb7c0053019230e6fad150e1841"
Oct 30 19:23:18.863288 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.863272    5820 scope.go:122] "RemoveContainer" containerID="56344f47feeeb1209c55b56f1b0503ab319b3eb7c0053019230e6fad150e1841"
Oct 30 19:23:18.863575 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:23:18.863560    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56344f47feeeb1209c55b56f1b0503ab319b3eb7c0053019230e6fad150e1841\": container with ID starting with 56344f47feeeb1209c55b56f1b0503ab319b3eb7c0053019230e6fad150e1841 not found: ID does not exist" containerID="56344f47feeeb1209c55b56f1b0503ab319b3eb7c0053019230e6fad150e1841"
Oct 30 19:23:18.863639 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.863582    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56344f47feeeb1209c55b56f1b0503ab319b3eb7c0053019230e6fad150e1841"} err="failed to get container status \"56344f47feeeb1209c55b56f1b0503ab319b3eb7c0053019230e6fad150e1841\": rpc error: code = NotFound desc = could not find container \"56344f47feeeb1209c55b56f1b0503ab319b3eb7c0053019230e6fad150e1841\": container with ID starting with 56344f47feeeb1209c55b56f1b0503ab319b3eb7c0053019230e6fad150e1841 not found: ID does not exist"
Oct 30 19:23:18.935659 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.935625    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-podinfo\") pod \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\" (UID: \"2c64155a-21e5-4b7c-99f9-f2d35fc457b0\") "
Oct 30 19:23:18.936009 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.935989    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq8bv\" (UniqueName: \"kubernetes.io/projected/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-kube-api-access-sq8bv\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:23:18.936009 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.936007    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:23:18.936148 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.936018    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:23:18.936148 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.936027    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:23:18.936148 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.936035    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:23:18.937872 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:18.937849    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-podinfo" pod "2c64155a-21e5-4b7c-99f9-f2d35fc457b0" (UID: "2c64155a-21e5-4b7c-99f9-f2d35fc457b0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:23:19.037127 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:19.037090    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c64155a-21e5-4b7c-99f9-f2d35fc457b0-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:23:19.265307 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:19.265275    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"]
Oct 30 19:23:19.271718 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:19.271694    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-k7sgm"]
Oct 30 19:23:19.437374 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:19.437294    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c64155a-21e5-4b7c-99f9-f2d35fc457b0" path="/var/lib/kubelet/pods/2c64155a-21e5-4b7c-99f9-f2d35fc457b0/volumes"
Oct 30 19:23:20.343885 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:20.343848    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:21.144797 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:21.144666    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:25.609348 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:25.609307    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:30.547345 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:30.547302    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:35.402607 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:35.402567    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:38.203406 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:23:38.203372    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:23:40.467875 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:40.467834    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:44.171187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:44.171151    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:45.669996 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:45.669916    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:50.260127 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.260071    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"]
Oct 30 19:23:50.260728 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.260714    5820 cpu_manager.go:414] "containerMap: removing container" podUID="2c64155a-21e5-4b7c-99f9-f2d35fc457b0" containerName="istio-proxy"
Oct 30 19:23:50.260775 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.260729    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c64155a-21e5-4b7c-99f9-f2d35fc457b0" containerName="istio-proxy"
Oct 30 19:23:50.260852 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.260843    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c64155a-21e5-4b7c-99f9-f2d35fc457b0" containerName="istio-proxy"
Oct 30 19:23:50.265438 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.265422    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"
Oct 30 19:23:50.318013 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.317989    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-c60f5c-serving-cert\""
Oct 30 19:23:50.350863 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.350828    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6706295-66df-48cf-a8da-e4b02e724aaa-maas-api-tls\") pod \"maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c\" (UID: \"d6706295-66df-48cf-a8da-e4b02e724aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"
Oct 30 19:23:50.351015 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.350886    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6706295-66df-48cf-a8da-e4b02e724aaa-openshift-service-ca\") pod \"maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c\" (UID: \"d6706295-66df-48cf-a8da-e4b02e724aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"
Oct 30 19:23:50.351108 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.351080    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7nmc\" (UniqueName: \"kubernetes.io/projected/d6706295-66df-48cf-a8da-e4b02e724aaa-kube-api-access-t7nmc\") pod \"maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c\" (UID: \"d6706295-66df-48cf-a8da-e4b02e724aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"
Oct 30 19:23:50.351239 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.351220    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6706295-66df-48cf-a8da-e4b02e724aaa-metrics-certs\") pod \"maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c\" (UID: \"d6706295-66df-48cf-a8da-e4b02e724aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"
Oct 30 19:23:50.370132 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.370100    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"]
Oct 30 19:23:50.452589 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.452557    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6706295-66df-48cf-a8da-e4b02e724aaa-maas-api-tls\") pod \"maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c\" (UID: \"d6706295-66df-48cf-a8da-e4b02e724aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"
Oct 30 19:23:50.452805 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.452609    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6706295-66df-48cf-a8da-e4b02e724aaa-openshift-service-ca\") pod \"maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c\" (UID: \"d6706295-66df-48cf-a8da-e4b02e724aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"
Oct 30 19:23:50.452805 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.452685    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7nmc\" (UniqueName: \"kubernetes.io/projected/d6706295-66df-48cf-a8da-e4b02e724aaa-kube-api-access-t7nmc\") pod \"maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c\" (UID: \"d6706295-66df-48cf-a8da-e4b02e724aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"
Oct 30 19:23:50.452805 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:23:50.452735    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-serving-cert: secret "maas-api-e2e-autores-c60f5c-serving-cert" not found
Oct 30 19:23:50.452805 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.452764    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6706295-66df-48cf-a8da-e4b02e724aaa-metrics-certs\") pod \"maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c\" (UID: \"d6706295-66df-48cf-a8da-e4b02e724aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"
Oct 30 19:23:50.453070 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:23:50.452849    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6706295-66df-48cf-a8da-e4b02e724aaa-maas-api-tls podName:d6706295-66df-48cf-a8da-e4b02e724aaa nodeName:}" failed. No retries permitted until 2026-10-30 19:23:50.952813708 +0000 UTC m=+2951.664562405 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d6706295-66df-48cf-a8da-e4b02e724aaa-maas-api-tls") pod "maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c" (UID: "d6706295-66df-48cf-a8da-e4b02e724aaa") : secret "maas-api-e2e-autores-c60f5c-serving-cert" not found
Oct 30 19:23:50.453450 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.453430    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6706295-66df-48cf-a8da-e4b02e724aaa-openshift-service-ca\") pod \"maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c\" (UID: \"d6706295-66df-48cf-a8da-e4b02e724aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"
Oct 30 19:23:50.455543 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.455520    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6706295-66df-48cf-a8da-e4b02e724aaa-metrics-certs\") pod \"maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c\" (UID: \"d6706295-66df-48cf-a8da-e4b02e724aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"
Oct 30 19:23:50.499151 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.499118    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:50.514828 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.514759    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7nmc\" (UniqueName: \"kubernetes.io/projected/d6706295-66df-48cf-a8da-e4b02e724aaa-kube-api-access-t7nmc\") pod \"maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c\" (UID: \"d6706295-66df-48cf-a8da-e4b02e724aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"
Oct 30 19:23:50.694679 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.687453    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:50.874193 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.874154    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:50.963221 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.963187    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6706295-66df-48cf-a8da-e4b02e724aaa-maas-api-tls\") pod \"maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c\" (UID: \"d6706295-66df-48cf-a8da-e4b02e724aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"
Oct 30 19:23:50.965825 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:50.965802    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6706295-66df-48cf-a8da-e4b02e724aaa-maas-api-tls\") pod \"maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c\" (UID: \"d6706295-66df-48cf-a8da-e4b02e724aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"
Oct 30 19:23:51.180096 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:51.180011    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"
Oct 30 19:23:51.558648 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:51.558617    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"]
Oct 30 19:23:51.560593 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:23:51.560572    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6706295_66df_48cf_a8da_e4b02e724aaa.slice/crio-78bd9f94d414668976b19f5d0a85de6414897204d1be46277123b99191581821 WatchSource:0}: Error finding container 78bd9f94d414668976b19f5d0a85de6414897204d1be46277123b99191581821: Status 404 returned error can't find the container with id 78bd9f94d414668976b19f5d0a85de6414897204d1be46277123b99191581821
Oct 30 19:23:51.562380 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:51.562361    5820 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:23:52.054095 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:52.054060    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c" event={"ID":"d6706295-66df-48cf-a8da-e4b02e724aaa","Type":"ContainerStarted","Data":"78bd9f94d414668976b19f5d0a85de6414897204d1be46277123b99191581821"}
Oct 30 19:23:53.061261 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:53.061226    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c" event={"ID":"d6706295-66df-48cf-a8da-e4b02e724aaa","Type":"ContainerStarted","Data":"aeebc5bea6eb43616c6f0ece1e8c2a97d547e1b10754ad23a39c442afb515bea"}
Oct 30 19:23:53.061638 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:53.061398    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"
Oct 30 19:23:53.178840 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:53.178779    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c" podStartSLOduration=2.175768329 podStartE2EDuration="3.178765087s" totalImagesPullingTime="1.002996758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:23:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:23:53.178549108 +0000 UTC m=+2953.890297809" watchObservedRunningTime="2026-10-30 19:23:53.178765087 +0000 UTC m=+2953.890513783"
Oct 30 19:23:55.546621 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:55.546571    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:57.219694 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:57.219664    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"]
Oct 30 19:23:57.220241 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:57.219864    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d" podUID="166098fd-c83d-4ce8-9b4c-9a69751e1e17" containerName="istio-proxy" containerID="cri-o://d044d9298c85a5548a0995a4a0cc9e11de550d33e5fcd0b83967f61657af9777" gracePeriod=30
Oct 30 19:23:57.269640 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:57.269602    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"]
Oct 30 19:23:57.269879 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:57.269852    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4" podUID="25b788f8-ea78-4bb2-a154-b30f48496c04" containerName="istio-proxy" containerID="cri-o://10bb75991d3d362be456807ea12c292e6cbc78ba8336bf8ae77c87d86a263c0a" gracePeriod=30
Oct 30 19:23:57.272498 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:57.272472    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"]
Oct 30 19:23:57.272704 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:57.272680    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd" podUID="4cbca4a3-6539-477b-a858-5b563ebd3bfb" containerName="istio-proxy" containerID="cri-o://ec1868e380d4afc51c4ffdb19c55ea43135433bc167c99ddbce69d6a376618c6" gracePeriod=30
Oct 30 19:23:59.072639 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:23:59.072604    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"
Oct 30 19:24:00.448646 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:00.448607    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:01.423191 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:01.423152    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:24:01.423379 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:01.423214    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d" podUID="166098fd-c83d-4ce8-9b4c-9a69751e1e17" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:24:02.352975 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.352932    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 30 19:24:02.353353 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.352992    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd" podUID="4cbca4a3-6539-477b-a858-5b563ebd3bfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 30 19:24:02.546681 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.546654    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:24:02.579162 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.579136    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:24:02.582446 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.582425    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:24:02.717149 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.717117    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istiod-ca-cert\") pod \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") "
Oct 30 19:24:02.717344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.717158    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-workload-socket\") pod \"25b788f8-ea78-4bb2-a154-b30f48496c04\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") "
Oct 30 19:24:02.717344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.717205    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-credential-socket\") pod \"25b788f8-ea78-4bb2-a154-b30f48496c04\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") "
Oct 30 19:24:02.717344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.717234    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4cbca4a3-6539-477b-a858-5b563ebd3bfb-kube-api-access-q6r6j\" (UniqueName: \"kubernetes.io/projected/4cbca4a3-6539-477b-a858-5b563ebd3bfb-kube-api-access-q6r6j\") pod \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") "
Oct 30 19:24:02.717344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.717331    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-token\" (UniqueName: \"kubernetes.io/projected/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-token\") pod \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") "
Oct 30 19:24:02.717896 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.717363    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-data\") pod \"25b788f8-ea78-4bb2-a154-b30f48496c04\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") "
Oct 30 19:24:02.717896 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.717426    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-envoy\") pod \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") "
Oct 30 19:24:02.717896 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.717455    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-envoy\") pod \"25b788f8-ea78-4bb2-a154-b30f48496c04\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") "
Oct 30 19:24:02.717896 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.717462    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-workload-socket" pod "25b788f8-ea78-4bb2-a154-b30f48496c04" (UID: "25b788f8-ea78-4bb2-a154-b30f48496c04"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:02.717896 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.717485    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-token\" (UniqueName: \"kubernetes.io/projected/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-token\") pod \"25b788f8-ea78-4bb2-a154-b30f48496c04\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") "
Oct 30 19:24:02.717896 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.717520    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istiod-ca-cert\") pod \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") "
Oct 30 19:24:02.717896 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.717563    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/166098fd-c83d-4ce8-9b4c-9a69751e1e17-kube-api-access-d6v9f\" (UniqueName: \"kubernetes.io/projected/166098fd-c83d-4ce8-9b4c-9a69751e1e17-kube-api-access-d6v9f\") pod \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") "
Oct 30 19:24:02.717896 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.717595    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-podinfo\") pod \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") "
Oct 30 19:24:02.717896 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.717624    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-workload-certs\") pod \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") "
Oct 30 19:24:02.718484 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.718231    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-credential-socket" pod "25b788f8-ea78-4bb2-a154-b30f48496c04" (UID: "25b788f8-ea78-4bb2-a154-b30f48496c04"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:02.718484 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.718263    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istiod-ca-cert" pod "166098fd-c83d-4ce8-9b4c-9a69751e1e17" (UID: "166098fd-c83d-4ce8-9b4c-9a69751e1e17"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:24:02.718484 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.718274    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-data\") pod \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") "
Oct 30 19:24:02.718677 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.718631    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-workload-socket\") pod \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") "
Oct 30 19:24:02.718999 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.718970    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-data" pod "25b788f8-ea78-4bb2-a154-b30f48496c04" (UID: "25b788f8-ea78-4bb2-a154-b30f48496c04"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:02.719461 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.719431    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-workload-certs" pod "4cbca4a3-6539-477b-a858-5b563ebd3bfb" (UID: "4cbca4a3-6539-477b-a858-5b563ebd3bfb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:02.719898 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.719871    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-data" pod "4cbca4a3-6539-477b-a858-5b563ebd3bfb" (UID: "4cbca4a3-6539-477b-a858-5b563ebd3bfb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:02.720019 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.719794    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istiod-ca-cert" pod "4cbca4a3-6539-477b-a858-5b563ebd3bfb" (UID: "4cbca4a3-6539-477b-a858-5b563ebd3bfb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:24:02.720154 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.720132    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-workload-socket" pod "4cbca4a3-6539-477b-a858-5b563ebd3bfb" (UID: "4cbca4a3-6539-477b-a858-5b563ebd3bfb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:02.720220 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.720202    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-token" pod "166098fd-c83d-4ce8-9b4c-9a69751e1e17" (UID: "166098fd-c83d-4ce8-9b4c-9a69751e1e17"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:24:02.721753 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.721706    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cbca4a3-6539-477b-a858-5b563ebd3bfb-kube-api-access-q6r6j" pod "4cbca4a3-6539-477b-a858-5b563ebd3bfb" (UID: "4cbca4a3-6539-477b-a858-5b563ebd3bfb"). InnerVolumeSpecName "kube-api-access-q6r6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:24:02.722320 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.721908    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-envoy" pod "25b788f8-ea78-4bb2-a154-b30f48496c04" (UID: "25b788f8-ea78-4bb2-a154-b30f48496c04"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:02.722320 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.721932    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-podinfo\") pod \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") "
Oct 30 19:24:02.722320 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.722059    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-data\") pod \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") "
Oct 30 19:24:02.722320 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.722100    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-credential-socket\") pod \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") "
Oct 30 19:24:02.722320 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.722134    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-workload-certs\") pod \"25b788f8-ea78-4bb2-a154-b30f48496c04\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") "
Oct 30 19:24:02.722320 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.722141    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-envoy" pod "4cbca4a3-6539-477b-a858-5b563ebd3bfb" (UID: "4cbca4a3-6539-477b-a858-5b563ebd3bfb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:02.722320 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.722180    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-credential-socket\") pod \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") "
Oct 30 19:24:02.722320 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.722235    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-workload-certs\") pod \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") "
Oct 30 19:24:02.722320 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.722262    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/25b788f8-ea78-4bb2-a154-b30f48496c04-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25b788f8-ea78-4bb2-a154-b30f48496c04-istiod-ca-cert\") pod \"25b788f8-ea78-4bb2-a154-b30f48496c04\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") "
Oct 30 19:24:02.723777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.722448    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-credential-socket" pod "166098fd-c83d-4ce8-9b4c-9a69751e1e17" (UID: "166098fd-c83d-4ce8-9b4c-9a69751e1e17"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:02.723777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.722506    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/166098fd-c83d-4ce8-9b4c-9a69751e1e17-kube-api-access-d6v9f" pod "166098fd-c83d-4ce8-9b4c-9a69751e1e17" (UID: "166098fd-c83d-4ce8-9b4c-9a69751e1e17"). InnerVolumeSpecName "kube-api-access-d6v9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:24:02.723777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.722665    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-workload-certs" pod "25b788f8-ea78-4bb2-a154-b30f48496c04" (UID: "25b788f8-ea78-4bb2-a154-b30f48496c04"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:02.723777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.722738    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-workload-certs" pod "166098fd-c83d-4ce8-9b4c-9a69751e1e17" (UID: "166098fd-c83d-4ce8-9b4c-9a69751e1e17"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:02.723777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.722802    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-credential-socket" pod "4cbca4a3-6539-477b-a858-5b563ebd3bfb" (UID: "4cbca4a3-6539-477b-a858-5b563ebd3bfb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:02.723777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.722969    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-envoy\") pod \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") "
Oct 30 19:24:02.723777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723013    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-token\" (UniqueName: \"kubernetes.io/projected/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-token\") pod \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\" (UID: \"4cbca4a3-6539-477b-a858-5b563ebd3bfb\") "
Oct 30 19:24:02.723777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723049    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-workload-socket\") pod \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\" (UID: \"166098fd-c83d-4ce8-9b4c-9a69751e1e17\") "
Oct 30 19:24:02.723777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723084    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-podinfo\") pod \"25b788f8-ea78-4bb2-a154-b30f48496c04\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") "
Oct 30 19:24:02.723777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723087    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25b788f8-ea78-4bb2-a154-b30f48496c04-istiod-ca-cert" pod "25b788f8-ea78-4bb2-a154-b30f48496c04" (UID: "25b788f8-ea78-4bb2-a154-b30f48496c04"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:24:02.723777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723133    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25b788f8-ea78-4bb2-a154-b30f48496c04-kube-api-access-qsjl7\" (UniqueName: \"kubernetes.io/projected/25b788f8-ea78-4bb2-a154-b30f48496c04-kube-api-access-qsjl7\") pod \"25b788f8-ea78-4bb2-a154-b30f48496c04\" (UID: \"25b788f8-ea78-4bb2-a154-b30f48496c04\") "
Oct 30 19:24:02.723777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723686    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.723777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723705    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.723777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723719    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.723777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723733    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.723777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723747    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25b788f8-ea78-4bb2-a154-b30f48496c04-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.723777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723761    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.723777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723774    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.723777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723788    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.725144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723803    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6r6j\" (UniqueName: \"kubernetes.io/projected/4cbca4a3-6539-477b-a858-5b563ebd3bfb-kube-api-access-q6r6j\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.725144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723817    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.725144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723830    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.725144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723843    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.725144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723856    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.725144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723868    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.725144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723880    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6v9f\" (UniqueName: \"kubernetes.io/projected/166098fd-c83d-4ce8-9b4c-9a69751e1e17-kube-api-access-d6v9f\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.725144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723892    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.725144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723905    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.725144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.723917    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cbca4a3-6539-477b-a858-5b563ebd3bfb-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.725144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.724108    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-workload-socket" pod "166098fd-c83d-4ce8-9b4c-9a69751e1e17" (UID: "166098fd-c83d-4ce8-9b4c-9a69751e1e17"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:02.725144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.724585    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-token" pod "25b788f8-ea78-4bb2-a154-b30f48496c04" (UID: "25b788f8-ea78-4bb2-a154-b30f48496c04"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:24:02.725670 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.725321    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-data" pod "166098fd-c83d-4ce8-9b4c-9a69751e1e17" (UID: "166098fd-c83d-4ce8-9b4c-9a69751e1e17"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:02.725670 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.725558    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-token" pod "4cbca4a3-6539-477b-a858-5b563ebd3bfb" (UID: "4cbca4a3-6539-477b-a858-5b563ebd3bfb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:24:02.726591 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.726563    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-podinfo" pod "166098fd-c83d-4ce8-9b4c-9a69751e1e17" (UID: "166098fd-c83d-4ce8-9b4c-9a69751e1e17"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:24:02.727062 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.727039    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-podinfo" pod "25b788f8-ea78-4bb2-a154-b30f48496c04" (UID: "25b788f8-ea78-4bb2-a154-b30f48496c04"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:24:02.727188 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.727144    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25b788f8-ea78-4bb2-a154-b30f48496c04-kube-api-access-qsjl7" pod "25b788f8-ea78-4bb2-a154-b30f48496c04" (UID: "25b788f8-ea78-4bb2-a154-b30f48496c04"). InnerVolumeSpecName "kube-api-access-qsjl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:24:02.727343 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.727320    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-envoy" pod "166098fd-c83d-4ce8-9b4c-9a69751e1e17" (UID: "166098fd-c83d-4ce8-9b4c-9a69751e1e17"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:02.727705 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.727687    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-podinfo" pod "4cbca4a3-6539-477b-a858-5b563ebd3bfb" (UID: "4cbca4a3-6539-477b-a858-5b563ebd3bfb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:24:02.825317 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.825288    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.825317 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.825318    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.825523 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.825328    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.825523 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.825338    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.825523 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.825353    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.825523 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.825364    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cbca4a3-6539-477b-a858-5b563ebd3bfb-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.825523 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.825373    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/166098fd-c83d-4ce8-9b4c-9a69751e1e17-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.825523 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.825381    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25b788f8-ea78-4bb2-a154-b30f48496c04-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:02.825523 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:02.825390    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsjl7\" (UniqueName: \"kubernetes.io/projected/25b788f8-ea78-4bb2-a154-b30f48496c04-kube-api-access-qsjl7\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:03.130402 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.130320    5820 generic.go:356] "Generic (PLEG): container finished" podID="25b788f8-ea78-4bb2-a154-b30f48496c04" containerID="10bb75991d3d362be456807ea12c292e6cbc78ba8336bf8ae77c87d86a263c0a" exitCode=0
Oct 30 19:24:03.130402 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.130376    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"
Oct 30 19:24:03.130627 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.130403    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4" event={"ID":"25b788f8-ea78-4bb2-a154-b30f48496c04","Type":"ContainerDied","Data":"10bb75991d3d362be456807ea12c292e6cbc78ba8336bf8ae77c87d86a263c0a"}
Oct 30 19:24:03.130627 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.130489    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4" event={"ID":"25b788f8-ea78-4bb2-a154-b30f48496c04","Type":"ContainerDied","Data":"6670c3aa01a922e1c7ca7f1a8522d1e62f2a1726d29a0d8730f54452538e5dd1"}
Oct 30 19:24:03.130627 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.130525    5820 scope.go:122] "RemoveContainer" containerID="10bb75991d3d362be456807ea12c292e6cbc78ba8336bf8ae77c87d86a263c0a"
Oct 30 19:24:03.132040 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.132020    5820 generic.go:356] "Generic (PLEG): container finished" podID="166098fd-c83d-4ce8-9b4c-9a69751e1e17" containerID="d044d9298c85a5548a0995a4a0cc9e11de550d33e5fcd0b83967f61657af9777" exitCode=0
Oct 30 19:24:03.132166 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.132084    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d" event={"ID":"166098fd-c83d-4ce8-9b4c-9a69751e1e17","Type":"ContainerDied","Data":"d044d9298c85a5548a0995a4a0cc9e11de550d33e5fcd0b83967f61657af9777"}
Oct 30 19:24:03.132166 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.132099    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"
Oct 30 19:24:03.132166 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.132156    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d" event={"ID":"166098fd-c83d-4ce8-9b4c-9a69751e1e17","Type":"ContainerDied","Data":"be8d5127b6436a439ddb7458002641ff000a8284bdfa1c58f4530bfc8aa9d3aa"}
Oct 30 19:24:03.133508 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.133488    5820 generic.go:356] "Generic (PLEG): container finished" podID="4cbca4a3-6539-477b-a858-5b563ebd3bfb" containerID="ec1868e380d4afc51c4ffdb19c55ea43135433bc167c99ddbce69d6a376618c6" exitCode=0
Oct 30 19:24:03.133620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.133603    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd" event={"ID":"4cbca4a3-6539-477b-a858-5b563ebd3bfb","Type":"ContainerDied","Data":"ec1868e380d4afc51c4ffdb19c55ea43135433bc167c99ddbce69d6a376618c6"}
Oct 30 19:24:03.133671 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.133641    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd" event={"ID":"4cbca4a3-6539-477b-a858-5b563ebd3bfb","Type":"ContainerDied","Data":"d277764d5333faa0b8fd4b3eafde71a2b6372317d8d414d1c5fd534cc5aadce6"}
Oct 30 19:24:03.133713 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.133708    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"
Oct 30 19:24:03.142318 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.142300    5820 scope.go:122] "RemoveContainer" containerID="10bb75991d3d362be456807ea12c292e6cbc78ba8336bf8ae77c87d86a263c0a"
Oct 30 19:24:03.142798 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:24:03.142653    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10bb75991d3d362be456807ea12c292e6cbc78ba8336bf8ae77c87d86a263c0a\": container with ID starting with 10bb75991d3d362be456807ea12c292e6cbc78ba8336bf8ae77c87d86a263c0a not found: ID does not exist" containerID="10bb75991d3d362be456807ea12c292e6cbc78ba8336bf8ae77c87d86a263c0a"
Oct 30 19:24:03.142798 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.142714    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10bb75991d3d362be456807ea12c292e6cbc78ba8336bf8ae77c87d86a263c0a"} err="failed to get container status \"10bb75991d3d362be456807ea12c292e6cbc78ba8336bf8ae77c87d86a263c0a\": rpc error: code = NotFound desc = could not find container \"10bb75991d3d362be456807ea12c292e6cbc78ba8336bf8ae77c87d86a263c0a\": container with ID starting with 10bb75991d3d362be456807ea12c292e6cbc78ba8336bf8ae77c87d86a263c0a not found: ID does not exist"
Oct 30 19:24:03.142798 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.142741    5820 scope.go:122] "RemoveContainer" containerID="d044d9298c85a5548a0995a4a0cc9e11de550d33e5fcd0b83967f61657af9777"
Oct 30 19:24:03.153035 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.153017    5820 scope.go:122] "RemoveContainer" containerID="d044d9298c85a5548a0995a4a0cc9e11de550d33e5fcd0b83967f61657af9777"
Oct 30 19:24:03.153301 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:24:03.153283    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d044d9298c85a5548a0995a4a0cc9e11de550d33e5fcd0b83967f61657af9777\": container with ID starting with d044d9298c85a5548a0995a4a0cc9e11de550d33e5fcd0b83967f61657af9777 not found: ID does not exist" containerID="d044d9298c85a5548a0995a4a0cc9e11de550d33e5fcd0b83967f61657af9777"
Oct 30 19:24:03.153354 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.153308    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d044d9298c85a5548a0995a4a0cc9e11de550d33e5fcd0b83967f61657af9777"} err="failed to get container status \"d044d9298c85a5548a0995a4a0cc9e11de550d33e5fcd0b83967f61657af9777\": rpc error: code = NotFound desc = could not find container \"d044d9298c85a5548a0995a4a0cc9e11de550d33e5fcd0b83967f61657af9777\": container with ID starting with d044d9298c85a5548a0995a4a0cc9e11de550d33e5fcd0b83967f61657af9777 not found: ID does not exist"
Oct 30 19:24:03.153354 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.153325    5820 scope.go:122] "RemoveContainer" containerID="ec1868e380d4afc51c4ffdb19c55ea43135433bc167c99ddbce69d6a376618c6"
Oct 30 19:24:03.162242 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.162220    5820 scope.go:122] "RemoveContainer" containerID="ec1868e380d4afc51c4ffdb19c55ea43135433bc167c99ddbce69d6a376618c6"
Oct 30 19:24:03.162525 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:24:03.162507    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec1868e380d4afc51c4ffdb19c55ea43135433bc167c99ddbce69d6a376618c6\": container with ID starting with ec1868e380d4afc51c4ffdb19c55ea43135433bc167c99ddbce69d6a376618c6 not found: ID does not exist" containerID="ec1868e380d4afc51c4ffdb19c55ea43135433bc167c99ddbce69d6a376618c6"
Oct 30 19:24:03.162588 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.162531    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec1868e380d4afc51c4ffdb19c55ea43135433bc167c99ddbce69d6a376618c6"} err="failed to get container status \"ec1868e380d4afc51c4ffdb19c55ea43135433bc167c99ddbce69d6a376618c6\": rpc error: code = NotFound desc = could not find container \"ec1868e380d4afc51c4ffdb19c55ea43135433bc167c99ddbce69d6a376618c6\": container with ID starting with ec1868e380d4afc51c4ffdb19c55ea43135433bc167c99ddbce69d6a376618c6 not found: ID does not exist"
Oct 30 19:24:03.281631 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.281602    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:03.296103 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.296075    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"]
Oct 30 19:24:03.301148 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.301128    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-6jmsd"]
Oct 30 19:24:03.411607 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.411533    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"]
Oct 30 19:24:03.419146 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.419118    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-cb4x4"]
Oct 30 19:24:03.437318 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.437290    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="25b788f8-ea78-4bb2-a154-b30f48496c04" path="/var/lib/kubelet/pods/25b788f8-ea78-4bb2-a154-b30f48496c04/volumes"
Oct 30 19:24:03.437746 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.437733    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4cbca4a3-6539-477b-a858-5b563ebd3bfb" path="/var/lib/kubelet/pods/4cbca4a3-6539-477b-a858-5b563ebd3bfb/volumes"
Oct 30 19:24:03.515674 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.515638    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:03.525588 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.525543    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"]
Oct 30 19:24:03.528472 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:03.528447    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-nd94d"]
Oct 30 19:24:05.413181 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:05.413146    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:05.437210 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:05.437176    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="166098fd-c83d-4ce8-9b4c-9a69751e1e17" path="/var/lib/kubelet/pods/166098fd-c83d-4ce8-9b4c-9a69751e1e17/volumes"
Oct 30 19:24:08.203723 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:24:08.203688    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:24:09.628563 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.628531    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"]
Oct 30 19:24:09.629168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.629154    5820 cpu_manager.go:414] "containerMap: removing container" podUID="166098fd-c83d-4ce8-9b4c-9a69751e1e17" containerName="istio-proxy"
Oct 30 19:24:09.629223 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.629169    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="166098fd-c83d-4ce8-9b4c-9a69751e1e17" containerName="istio-proxy"
Oct 30 19:24:09.629223 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.629184    5820 cpu_manager.go:414] "containerMap: removing container" podUID="25b788f8-ea78-4bb2-a154-b30f48496c04" containerName="istio-proxy"
Oct 30 19:24:09.629223 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.629189    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25b788f8-ea78-4bb2-a154-b30f48496c04" containerName="istio-proxy"
Oct 30 19:24:09.629223 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.629206    5820 cpu_manager.go:414] "containerMap: removing container" podUID="4cbca4a3-6539-477b-a858-5b563ebd3bfb" containerName="istio-proxy"
Oct 30 19:24:09.629223 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.629211    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4cbca4a3-6539-477b-a858-5b563ebd3bfb" containerName="istio-proxy"
Oct 30 19:24:09.629431 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.629296    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="166098fd-c83d-4ce8-9b4c-9a69751e1e17" containerName="istio-proxy"
Oct 30 19:24:09.629431 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.629306    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="25b788f8-ea78-4bb2-a154-b30f48496c04" containerName="istio-proxy"
Oct 30 19:24:09.629431 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.629315    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="4cbca4a3-6539-477b-a858-5b563ebd3bfb" containerName="istio-proxy"
Oct 30 19:24:09.632812 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.632795    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:09.702621 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.702589    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-home\") pod \"auth-test-model-23302c-kserve-58ff984649-8gpvk\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") " pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:09.702838 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.702631    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr9sj\" (UniqueName: \"kubernetes.io/projected/90c808d6-a5be-4e0e-979a-69a5cc4571fc-kube-api-access-nr9sj\") pod \"auth-test-model-23302c-kserve-58ff984649-8gpvk\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") " pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:09.702838 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.702783    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-dshm\") pod \"auth-test-model-23302c-kserve-58ff984649-8gpvk\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") " pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:09.703018 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.702846    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-tmp-dir\") pod \"auth-test-model-23302c-kserve-58ff984649-8gpvk\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") " pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:09.703096 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.703067    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90c808d6-a5be-4e0e-979a-69a5cc4571fc-tls-certs\") pod \"auth-test-model-23302c-kserve-58ff984649-8gpvk\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") " pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:09.703164 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.703113    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-model-cache\") pod \"auth-test-model-23302c-kserve-58ff984649-8gpvk\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") " pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:09.734533 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.734505    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w1-23302c\"/\"openshift-service-ca.crt\""
Oct 30 19:24:09.734673 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.734520    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w1-23302c\"/\"default-dockercfg-6pxpt\""
Oct 30 19:24:09.734673 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.734650    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w1-23302c\"/\"auth-test-model-23302c-kserve-self-signed-certs\""
Oct 30 19:24:09.734772 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.734752    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w1-23302c\"/\"kube-root-ca.crt\""
Oct 30 19:24:09.737609 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.737585    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"]
Oct 30 19:24:09.800165 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.800127    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:09.804977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.804899    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90c808d6-a5be-4e0e-979a-69a5cc4571fc-tls-certs\") pod \"auth-test-model-23302c-kserve-58ff984649-8gpvk\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") " pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:09.805267 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.805230    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-model-cache\") pod \"auth-test-model-23302c-kserve-58ff984649-8gpvk\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") " pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:09.805420 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.805403    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-home\") pod \"auth-test-model-23302c-kserve-58ff984649-8gpvk\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") " pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:09.805557 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.805540    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr9sj\" (UniqueName: \"kubernetes.io/projected/90c808d6-a5be-4e0e-979a-69a5cc4571fc-kube-api-access-nr9sj\") pod \"auth-test-model-23302c-kserve-58ff984649-8gpvk\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") " pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:09.805768 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.805750    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-dshm\") pod \"auth-test-model-23302c-kserve-58ff984649-8gpvk\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") " pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:09.805921 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.805897    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-tmp-dir\") pod \"auth-test-model-23302c-kserve-58ff984649-8gpvk\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") " pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:09.806660 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.806619    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-tmp-dir\") pod \"auth-test-model-23302c-kserve-58ff984649-8gpvk\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") " pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:09.810125 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.807931    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-model-cache\") pod \"auth-test-model-23302c-kserve-58ff984649-8gpvk\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") " pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:09.810250 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.810041    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-home\") pod \"auth-test-model-23302c-kserve-58ff984649-8gpvk\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") " pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:09.811490 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.811466    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90c808d6-a5be-4e0e-979a-69a5cc4571fc-tls-certs\") pod \"auth-test-model-23302c-kserve-58ff984649-8gpvk\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") " pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:09.811490 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.811480    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-dshm\") pod \"auth-test-model-23302c-kserve-58ff984649-8gpvk\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") " pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:09.871276 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.871244    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr9sj\" (UniqueName: \"kubernetes.io/projected/90c808d6-a5be-4e0e-979a-69a5cc4571fc-kube-api-access-nr9sj\") pod \"auth-test-model-23302c-kserve-58ff984649-8gpvk\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") " pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:09.945081 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:09.945006    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:10.190489 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:10.190438    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:10.348012 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:10.347983    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"]
Oct 30 19:24:10.349399 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:24:10.349370    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90c808d6_a5be_4e0e_979a_69a5cc4571fc.slice/crio-d23ca2ed5d5b9453356a1006526793da258dc7736c25e2c07befe6e7f1795e15 WatchSource:0}: Error finding container d23ca2ed5d5b9453356a1006526793da258dc7736c25e2c07befe6e7f1795e15: Status 404 returned error can't find the container with id d23ca2ed5d5b9453356a1006526793da258dc7736c25e2c07befe6e7f1795e15
Oct 30 19:24:11.191000 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:11.190941    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk" event={"ID":"90c808d6-a5be-4e0e-979a-69a5cc4571fc","Type":"ContainerStarted","Data":"fc6d23923595885e81e058ee73df2cb01a2570cdc54d333e754a320f893777d8"}
Oct 30 19:24:11.191000 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:11.191005    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk" event={"ID":"90c808d6-a5be-4e0e-979a-69a5cc4571fc","Type":"ContainerStarted","Data":"d23ca2ed5d5b9453356a1006526793da258dc7736c25e2c07befe6e7f1795e15"}
Oct 30 19:24:11.310766 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:11.310707    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk" podStartSLOduration=2.310693572 podStartE2EDuration="2.310693572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:24:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:24:11.307195451 +0000 UTC m=+2972.018944147" watchObservedRunningTime="2026-10-30 19:24:11.310693572 +0000 UTC m=+2972.022442314"
Oct 30 19:24:11.946025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:11.945988    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:15.247845 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:15.247775    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:19.384875 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.384786    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"]
Oct 30 19:24:19.389694 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.389669    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:19.493896 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.493861    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-praxis-a-81d66e\"/\"openshift-service-ca.crt\""
Oct 30 19:24:19.493896 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.493876    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-praxis-a-81d66e\"/\"kube-root-ca.crt\""
Oct 30 19:24:19.494225 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.494210    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-praxis-a-81d66e\"/\"default-dockercfg-m2rvl\""
Oct 30 19:24:19.494338 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.494318    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-praxis-a-81d66e\"/\"praxis-route-81d66e-kserve-self-signed-certs\""
Oct 30 19:24:19.496600 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.496580    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"]
Oct 30 19:24:19.520516 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.520484    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:19.547001 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.545642    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6lhr\" (UniqueName: \"kubernetes.io/projected/354bfaa6-d937-437c-bcd9-48a7db460d23-kube-api-access-v6lhr\") pod \"praxis-route-81d66e-kserve-66bccdcc6f-kw4lg\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") " pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:19.547001 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.545733    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-model-cache\") pod \"praxis-route-81d66e-kserve-66bccdcc6f-kw4lg\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") " pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:19.547001 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.545778    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-home\") pod \"praxis-route-81d66e-kserve-66bccdcc6f-kw4lg\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") " pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:19.547001 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.545891    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-tmp-dir\") pod \"praxis-route-81d66e-kserve-66bccdcc6f-kw4lg\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") " pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:19.547001 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.545925    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-dshm\") pod \"praxis-route-81d66e-kserve-66bccdcc6f-kw4lg\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") " pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:19.548250 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.547582    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/354bfaa6-d937-437c-bcd9-48a7db460d23-tls-certs\") pod \"praxis-route-81d66e-kserve-66bccdcc6f-kw4lg\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") " pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:19.649112 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.649019    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6lhr\" (UniqueName: \"kubernetes.io/projected/354bfaa6-d937-437c-bcd9-48a7db460d23-kube-api-access-v6lhr\") pod \"praxis-route-81d66e-kserve-66bccdcc6f-kw4lg\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") " pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:19.649112 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.649099    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-model-cache\") pod \"praxis-route-81d66e-kserve-66bccdcc6f-kw4lg\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") " pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:19.649346 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.649147    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-home\") pod \"praxis-route-81d66e-kserve-66bccdcc6f-kw4lg\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") " pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:19.649346 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.649248    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-tmp-dir\") pod \"praxis-route-81d66e-kserve-66bccdcc6f-kw4lg\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") " pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:19.649346 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.649294    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-dshm\") pod \"praxis-route-81d66e-kserve-66bccdcc6f-kw4lg\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") " pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:19.649478 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.649340    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/354bfaa6-d937-437c-bcd9-48a7db460d23-tls-certs\") pod \"praxis-route-81d66e-kserve-66bccdcc6f-kw4lg\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") " pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:19.649606 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.649578    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-model-cache\") pod \"praxis-route-81d66e-kserve-66bccdcc6f-kw4lg\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") " pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:19.649699 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.649642    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-tmp-dir\") pod \"praxis-route-81d66e-kserve-66bccdcc6f-kw4lg\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") " pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:19.650010 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.649979    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-home\") pod \"praxis-route-81d66e-kserve-66bccdcc6f-kw4lg\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") " pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:19.652181 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.652154    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-dshm\") pod \"praxis-route-81d66e-kserve-66bccdcc6f-kw4lg\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") " pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:19.652551 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.652528    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/354bfaa6-d937-437c-bcd9-48a7db460d23-tls-certs\") pod \"praxis-route-81d66e-kserve-66bccdcc6f-kw4lg\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") " pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:19.718287 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.718258    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6lhr\" (UniqueName: \"kubernetes.io/projected/354bfaa6-d937-437c-bcd9-48a7db460d23-kube-api-access-v6lhr\") pod \"praxis-route-81d66e-kserve-66bccdcc6f-kw4lg\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") " pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:19.946246 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.946156    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:19.958813 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:19.958786    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:20.004135 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:20.004101    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:20.257130 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:20.257053    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:24:20.385039 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:24:20.385007    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod354bfaa6_d937_437c_bcd9_48a7db460d23.slice/crio-d9b274b7bb737a9dca7eb1330553ef444565622487c34821c761c5b6598b1e7c WatchSource:0}: Error finding container d9b274b7bb737a9dca7eb1330553ef444565622487c34821c761c5b6598b1e7c: Status 404 returned error can't find the container with id d9b274b7bb737a9dca7eb1330553ef444565622487c34821c761c5b6598b1e7c
Oct 30 19:24:20.386317 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:20.386293    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"]
Oct 30 19:24:20.522033 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:20.516513    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:21.254478 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:21.254437    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg" event={"ID":"354bfaa6-d937-437c-bcd9-48a7db460d23","Type":"ContainerStarted","Data":"c37ecdd6b8dacb56d871eb421c409c9612eb8079d8117b77cfe3057f29027264"}
Oct 30 19:24:21.254665 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:21.254489    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg" event={"ID":"354bfaa6-d937-437c-bcd9-48a7db460d23","Type":"ContainerStarted","Data":"d9b274b7bb737a9dca7eb1330553ef444565622487c34821c761c5b6598b1e7c"}
Oct 30 19:24:21.375304 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:21.375254    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg" podStartSLOduration=2.375241886 podStartE2EDuration="2.375241886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:24:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:24:21.37498518 +0000 UTC m=+2982.086733873" watchObservedRunningTime="2026-10-30 19:24:21.375241886 +0000 UTC m=+2982.086990583"
Oct 30 19:24:22.005243 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:22.005212    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:25.483870 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:25.483830    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:30.004306 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:30.004271    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:30.017223 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:30.017197    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:30.332275 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:30.332248    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:24:30.470851 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:30.470505    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:33.133135 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:33.133101    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:35.379321 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:35.379284    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:38.203541 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:24:38.203504    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:24:40.260890 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:40.260847    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:45.035884 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:45.035839    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:49.868073 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:49.868036    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:54.646791 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:54.646751    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:59.651990 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:24:59.651928    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:03.630447 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:25:03.630412    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:04.442451 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:25:04.442416    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:07.994188 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:25:07.994139    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:08.204349 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:25:08.204307    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:25:09.547972 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:25:09.547721    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:13.225241 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:25:13.225209    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:13.492533 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:25:13.492450    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:14.538701 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:25:14.538665    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:19.440728 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:25:19.440689    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:24.336841 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:25:24.336798    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:27.810756 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:25:27.810714    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:29.164705 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:25:29.164662    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:33.998895 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:25:33.998822    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:38.204787 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:25:38.204746    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:25:38.838426 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:25:38.838389    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:43.674399 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:25:43.674287    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:45.786627 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:25:45.786579    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:48.504016 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:25:48.503974    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:53.277476 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:25:53.277411    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:58.120100 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:25:58.120056    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:59.336915 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:25:59.336863    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:59.521497 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:25:59.521443    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:02.871235 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:02.871188    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:03.966914 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:03.966870    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:07.665846 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:07.665792    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:08.203866 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:26:08.203831    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:26:12.575774 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:12.575740    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:17.456149 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:17.455839    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:22.262019 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:22.261981    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:27.149191 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:27.149147    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:29.206136 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:29.206089    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:31.973255 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:31.973213    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:32.769575 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:32.769539    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:36.767446 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:36.767407    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:38.203931 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:26:38.203897    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:26:39.452437 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.452407    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"]
Oct 30 19:26:39.460538 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.460510    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:39.514660 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.514631    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"]
Oct 30 19:26:39.560880 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.560845    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w0-a56c7c\"/\"openshift-service-ca.crt\""
Oct 30 19:26:39.561075 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.560866    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w0-a56c7c\"/\"kube-root-ca.crt\""
Oct 30 19:26:39.562669 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.562643    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w0-a56c7c\"/\"default-dockercfg-hfphq\""
Oct 30 19:26:39.562669 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.562653    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w0-a56c7c\"/\"auth-test-model-a56c7c-kserve-self-signed-certs\""
Oct 30 19:26:39.585219 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.585190    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:39.629062 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.629025    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qt6m\" (UniqueName: \"kubernetes.io/projected/bfdd1d8f-7962-4456-9314-666d19e646d6-kube-api-access-9qt6m\") pod \"auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") " pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:39.629352 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.629329    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-home\") pod \"auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") " pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:39.629660 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.629624    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-dshm\") pod \"auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") " pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:39.629801 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.629781    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-model-cache\") pod \"auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") " pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:39.630063 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.630041    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-tmp-dir\") pod \"auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") " pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:39.630173 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.630076    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfdd1d8f-7962-4456-9314-666d19e646d6-tls-certs\") pod \"auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") " pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:39.731233 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.731144    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qt6m\" (UniqueName: \"kubernetes.io/projected/bfdd1d8f-7962-4456-9314-666d19e646d6-kube-api-access-9qt6m\") pod \"auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") " pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:39.731233 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.731180    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-home\") pod \"auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") " pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:39.731474 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.731252    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-dshm\") pod \"auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") " pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:39.731474 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.731399    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-model-cache\") pod \"auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") " pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:39.731474 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.731448    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-tmp-dir\") pod \"auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") " pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:39.731642 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.731546    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfdd1d8f-7962-4456-9314-666d19e646d6-tls-certs\") pod \"auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") " pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:39.731642 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.731610    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-home\") pod \"auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") " pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:39.731783 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.731760    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-model-cache\") pod \"auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") " pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:39.731871 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.731851    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-tmp-dir\") pod \"auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") " pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:39.733697 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.733671    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-dshm\") pod \"auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") " pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:39.734188 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.734172    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfdd1d8f-7962-4456-9314-666d19e646d6-tls-certs\") pod \"auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") " pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:39.791990 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:39.791966    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qt6m\" (UniqueName: \"kubernetes.io/projected/bfdd1d8f-7962-4456-9314-666d19e646d6-kube-api-access-9qt6m\") pod \"auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") " pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:40.074644 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:40.074617    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:40.453272 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:40.453242    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"]
Oct 30 19:26:40.454857 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:26:40.454828    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbfdd1d8f_7962_4456_9314_666d19e646d6.slice/crio-9b71d32f0e794aeb785a5e620deab53781b3babc4c8b3d49e33e24fe389b3e54 WatchSource:0}: Error finding container 9b71d32f0e794aeb785a5e620deab53781b3babc4c8b3d49e33e24fe389b3e54: Status 404 returned error can't find the container with id 9b71d32f0e794aeb785a5e620deab53781b3babc4c8b3d49e33e24fe389b3e54
Oct 30 19:26:41.161934 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:41.161895    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6" event={"ID":"bfdd1d8f-7962-4456-9314-666d19e646d6","Type":"ContainerStarted","Data":"827d33f54cfd631603daf7881e75721edb6d0397809945935f3e1a689745d7b8"}
Oct 30 19:26:41.162143 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:41.161961    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6" event={"ID":"bfdd1d8f-7962-4456-9314-666d19e646d6","Type":"ContainerStarted","Data":"9b71d32f0e794aeb785a5e620deab53781b3babc4c8b3d49e33e24fe389b3e54"}
Oct 30 19:26:41.281573 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:41.281512    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6" podStartSLOduration=2.281500073 podStartE2EDuration="2.281500073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:41.279010174 +0000 UTC m=+3121.990758872" watchObservedRunningTime="2026-10-30 19:26:41.281500073 +0000 UTC m=+3121.993248769"
Oct 30 19:26:41.846996 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:41.846425    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:42.075763 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:42.075732    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:46.948292 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:46.948252    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:50.075041 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:50.075001    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:50.087970 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:50.087927    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:50.229313 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:50.229283    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:26:50.371844 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:50.371769    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc"]
Oct 30 19:26:50.479471 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:50.479434    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n"]
Oct 30 19:26:50.838756 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:50.838717    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:51.020670 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:51.020637    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:51.720942 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:51.720910    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:52.417723 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:52.417688    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:52.645335 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:52.645302    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:55.372919 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:55.372881    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc" podUID="9dbe29c1-0a26-4859-a57c-460ddc518f24" containerName="payload-processing" containerID="cri-o://d5f0aa1d66542cfe2a59560d13635810afa31971f898cbe33506c341a4b8bd95" gracePeriod=25
Oct 30 19:26:55.480377 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:55.480339    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n" podUID="dba918a1-39a1-41ad-bb05-55a5c5f6981c" containerName="payload-pre-processing" containerID="cri-o://674640f6e10ac4d5ba3d303774ea5e7dc2398714504ff0c435bfbacd59ecdbac" gracePeriod=25
Oct 30 19:26:56.561413 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:56.561368    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:56.890235 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:56.890141    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n" podUID="dba918a1-39a1-41ad-bb05-55a5c5f6981c" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:26:56.890235 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:26:56.890197    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc" podUID="9dbe29c1-0a26-4859-a57c-460ddc518f24" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:27:01.471963 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:01.471906    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:04.313869 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:04.313835    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:06.349789 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:06.349746    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:06.890196 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:06.890158    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc" podUID="9dbe29c1-0a26-4859-a57c-460ddc518f24" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:27:06.890389 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:06.890167    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n" podUID="dba918a1-39a1-41ad-bb05-55a5c5f6981c" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:27:08.203461 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:27:08.203422    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:27:11.133461 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:11.133419    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:13.901272 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:13.901240    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:14.108863 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:14.108824    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:15.657769 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:15.657743    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc"
Oct 30 19:27:15.751331 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:15.751302    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9dbe29c1-0a26-4859-a57c-460ddc518f24-kube-api-access-q558k\" (UniqueName: \"kubernetes.io/projected/9dbe29c1-0a26-4859-a57c-460ddc518f24-kube-api-access-q558k\") pod \"9dbe29c1-0a26-4859-a57c-460ddc518f24\" (UID: \"9dbe29c1-0a26-4859-a57c-460ddc518f24\") "
Oct 30 19:27:15.751464 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:15.751448    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9dbe29c1-0a26-4859-a57c-460ddc518f24-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dbe29c1-0a26-4859-a57c-460ddc518f24-plugins-config-volume\") pod \"9dbe29c1-0a26-4859-a57c-460ddc518f24\" (UID: \"9dbe29c1-0a26-4859-a57c-460ddc518f24\") "
Oct 30 19:27:15.751954 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:15.751911    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dbe29c1-0a26-4859-a57c-460ddc518f24-plugins-config-volume" pod "9dbe29c1-0a26-4859-a57c-460ddc518f24" (UID: "9dbe29c1-0a26-4859-a57c-460ddc518f24"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:27:15.753571 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:15.753546    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dbe29c1-0a26-4859-a57c-460ddc518f24-kube-api-access-q558k" pod "9dbe29c1-0a26-4859-a57c-460ddc518f24" (UID: "9dbe29c1-0a26-4859-a57c-460ddc518f24"). InnerVolumeSpecName "kube-api-access-q558k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:15.760458 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:15.760437    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n"
Oct 30 19:27:15.852472 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:15.852445    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dba918a1-39a1-41ad-bb05-55a5c5f6981c-kube-api-access-p45vr\" (UniqueName: \"kubernetes.io/projected/dba918a1-39a1-41ad-bb05-55a5c5f6981c-kube-api-access-p45vr\") pod \"dba918a1-39a1-41ad-bb05-55a5c5f6981c\" (UID: \"dba918a1-39a1-41ad-bb05-55a5c5f6981c\") "
Oct 30 19:27:15.852678 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:15.852586    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dba918a1-39a1-41ad-bb05-55a5c5f6981c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dba918a1-39a1-41ad-bb05-55a5c5f6981c-plugins-config-volume\") pod \"dba918a1-39a1-41ad-bb05-55a5c5f6981c\" (UID: \"dba918a1-39a1-41ad-bb05-55a5c5f6981c\") "
Oct 30 19:27:15.852977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:15.852941    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q558k\" (UniqueName: \"kubernetes.io/projected/9dbe29c1-0a26-4859-a57c-460ddc518f24-kube-api-access-q558k\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:15.853062 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:15.852983    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dbe29c1-0a26-4859-a57c-460ddc518f24-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:15.853062 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:15.853017    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dba918a1-39a1-41ad-bb05-55a5c5f6981c-plugins-config-volume" pod "dba918a1-39a1-41ad-bb05-55a5c5f6981c" (UID: "dba918a1-39a1-41ad-bb05-55a5c5f6981c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:27:15.854545 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:15.854520    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dba918a1-39a1-41ad-bb05-55a5c5f6981c-kube-api-access-p45vr" pod "dba918a1-39a1-41ad-bb05-55a5c5f6981c" (UID: "dba918a1-39a1-41ad-bb05-55a5c5f6981c"). InnerVolumeSpecName "kube-api-access-p45vr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:15.943006 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:15.942890    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:15.953659 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:15.953632    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dba918a1-39a1-41ad-bb05-55a5c5f6981c-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:15.953659 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:15.953659    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p45vr\" (UniqueName: \"kubernetes.io/projected/dba918a1-39a1-41ad-bb05-55a5c5f6981c-kube-api-access-p45vr\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:16.384251 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:16.384211    5820 generic.go:356] "Generic (PLEG): container finished" podID="9dbe29c1-0a26-4859-a57c-460ddc518f24" containerID="d5f0aa1d66542cfe2a59560d13635810afa31971f898cbe33506c341a4b8bd95" exitCode=0
Oct 30 19:27:16.384445 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:16.384254    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc"
Oct 30 19:27:16.384445 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:16.384298    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc" event={"ID":"9dbe29c1-0a26-4859-a57c-460ddc518f24","Type":"ContainerDied","Data":"d5f0aa1d66542cfe2a59560d13635810afa31971f898cbe33506c341a4b8bd95"}
Oct 30 19:27:16.384445 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:16.384356    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc" event={"ID":"9dbe29c1-0a26-4859-a57c-460ddc518f24","Type":"ContainerDied","Data":"c85c58cf9884e6546b39645edd2ce6bb4ca846e8c90df73c4f7e4686e1e519ff"}
Oct 30 19:27:16.384445 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:16.384396    5820 scope.go:122] "RemoveContainer" containerID="d5f0aa1d66542cfe2a59560d13635810afa31971f898cbe33506c341a4b8bd95"
Oct 30 19:27:16.386336 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:16.386310    5820 generic.go:356] "Generic (PLEG): container finished" podID="dba918a1-39a1-41ad-bb05-55a5c5f6981c" containerID="674640f6e10ac4d5ba3d303774ea5e7dc2398714504ff0c435bfbacd59ecdbac" exitCode=0
Oct 30 19:27:16.386477 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:16.386361    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n"
Oct 30 19:27:16.386477 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:16.386360    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n" event={"ID":"dba918a1-39a1-41ad-bb05-55a5c5f6981c","Type":"ContainerDied","Data":"674640f6e10ac4d5ba3d303774ea5e7dc2398714504ff0c435bfbacd59ecdbac"}
Oct 30 19:27:16.386600 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:16.386510    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n" event={"ID":"dba918a1-39a1-41ad-bb05-55a5c5f6981c","Type":"ContainerDied","Data":"44790806b242db41bb272d7623faf8296c6419ecf875dc498c480c653aced312"}
Oct 30 19:27:16.399641 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:16.399622    5820 scope.go:122] "RemoveContainer" containerID="d5f0aa1d66542cfe2a59560d13635810afa31971f898cbe33506c341a4b8bd95"
Oct 30 19:27:16.399896 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:27:16.399869    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5f0aa1d66542cfe2a59560d13635810afa31971f898cbe33506c341a4b8bd95\": container with ID starting with d5f0aa1d66542cfe2a59560d13635810afa31971f898cbe33506c341a4b8bd95 not found: ID does not exist" containerID="d5f0aa1d66542cfe2a59560d13635810afa31971f898cbe33506c341a4b8bd95"
Oct 30 19:27:16.399980 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:16.399895    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5f0aa1d66542cfe2a59560d13635810afa31971f898cbe33506c341a4b8bd95"} err="failed to get container status \"d5f0aa1d66542cfe2a59560d13635810afa31971f898cbe33506c341a4b8bd95\": rpc error: code = NotFound desc = could not find container \"d5f0aa1d66542cfe2a59560d13635810afa31971f898cbe33506c341a4b8bd95\": container with ID starting with d5f0aa1d66542cfe2a59560d13635810afa31971f898cbe33506c341a4b8bd95 not found: ID does not exist"
Oct 30 19:27:16.399980 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:16.399916    5820 scope.go:122] "RemoveContainer" containerID="674640f6e10ac4d5ba3d303774ea5e7dc2398714504ff0c435bfbacd59ecdbac"
Oct 30 19:27:16.409016 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:16.409000    5820 scope.go:122] "RemoveContainer" containerID="674640f6e10ac4d5ba3d303774ea5e7dc2398714504ff0c435bfbacd59ecdbac"
Oct 30 19:27:16.409292 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:27:16.409271    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"674640f6e10ac4d5ba3d303774ea5e7dc2398714504ff0c435bfbacd59ecdbac\": container with ID starting with 674640f6e10ac4d5ba3d303774ea5e7dc2398714504ff0c435bfbacd59ecdbac not found: ID does not exist" containerID="674640f6e10ac4d5ba3d303774ea5e7dc2398714504ff0c435bfbacd59ecdbac"
Oct 30 19:27:16.409354 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:16.409300    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"674640f6e10ac4d5ba3d303774ea5e7dc2398714504ff0c435bfbacd59ecdbac"} err="failed to get container status \"674640f6e10ac4d5ba3d303774ea5e7dc2398714504ff0c435bfbacd59ecdbac\": rpc error: code = NotFound desc = could not find container \"674640f6e10ac4d5ba3d303774ea5e7dc2398714504ff0c435bfbacd59ecdbac\": container with ID starting with 674640f6e10ac4d5ba3d303774ea5e7dc2398714504ff0c435bfbacd59ecdbac not found: ID does not exist"
Oct 30 19:27:16.550581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:16.550549    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n"]
Oct 30 19:27:16.569733 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:16.569697    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-81d66e-7bd4c44cf5-22t6n"]
Oct 30 19:27:16.664890 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:16.664832    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc"]
Oct 30 19:27:16.670698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:16.670676    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-81d66e-9c44b64d8-rqjqc"]
Oct 30 19:27:17.100181 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:17.100142    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:17.437323 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:17.437223    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9dbe29c1-0a26-4859-a57c-460ddc518f24" path="/var/lib/kubelet/pods/9dbe29c1-0a26-4859-a57c-460ddc518f24/volumes"
Oct 30 19:27:17.437600 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:17.437588    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dba918a1-39a1-41ad-bb05-55a5c5f6981c" path="/var/lib/kubelet/pods/dba918a1-39a1-41ad-bb05-55a5c5f6981c/volumes"
Oct 30 19:27:21.099864 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:21.099821    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:26.041582 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:26.041543    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:29.547510 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:29.547454    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:29.773042 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:29.773005    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:30.280316 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:30.280284    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"]
Oct 30 19:27:30.280551 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:30.280529    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c" podUID="d6706295-66df-48cf-a8da-e4b02e724aaa" containerName="maas-api" containerID="cri-o://aeebc5bea6eb43616c6f0ece1e8c2a97d547e1b10754ad23a39c442afb515bea" gracePeriod=30
Oct 30 19:27:30.441310 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:30.441269    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:30.477583 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:30.477547    5820 generic.go:356] "Generic (PLEG): container finished" podID="d6706295-66df-48cf-a8da-e4b02e724aaa" containerID="aeebc5bea6eb43616c6f0ece1e8c2a97d547e1b10754ad23a39c442afb515bea" exitCode=0
Oct 30 19:27:30.477734 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:30.477624    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c" event={"ID":"d6706295-66df-48cf-a8da-e4b02e724aaa","Type":"ContainerDied","Data":"aeebc5bea6eb43616c6f0ece1e8c2a97d547e1b10754ad23a39c442afb515bea"}
Oct 30 19:27:30.563366 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:30.563343    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"
Oct 30 19:27:30.619936 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:30.619911    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d6706295-66df-48cf-a8da-e4b02e724aaa-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6706295-66df-48cf-a8da-e4b02e724aaa-openshift-service-ca\") pod \"d6706295-66df-48cf-a8da-e4b02e724aaa\" (UID: \"d6706295-66df-48cf-a8da-e4b02e724aaa\") "
Oct 30 19:27:30.620086 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:30.619942    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d6706295-66df-48cf-a8da-e4b02e724aaa-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6706295-66df-48cf-a8da-e4b02e724aaa-metrics-certs\") pod \"d6706295-66df-48cf-a8da-e4b02e724aaa\" (UID: \"d6706295-66df-48cf-a8da-e4b02e724aaa\") "
Oct 30 19:27:30.620146 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:30.620081    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6706295-66df-48cf-a8da-e4b02e724aaa-kube-api-access-t7nmc\" (UniqueName: \"kubernetes.io/projected/d6706295-66df-48cf-a8da-e4b02e724aaa-kube-api-access-t7nmc\") pod \"d6706295-66df-48cf-a8da-e4b02e724aaa\" (UID: \"d6706295-66df-48cf-a8da-e4b02e724aaa\") "
Oct 30 19:27:30.620146 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:30.620134    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d6706295-66df-48cf-a8da-e4b02e724aaa-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6706295-66df-48cf-a8da-e4b02e724aaa-maas-api-tls\") pod \"d6706295-66df-48cf-a8da-e4b02e724aaa\" (UID: \"d6706295-66df-48cf-a8da-e4b02e724aaa\") "
Oct 30 19:27:30.620310 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:30.620278    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6706295-66df-48cf-a8da-e4b02e724aaa-openshift-service-ca" pod "d6706295-66df-48cf-a8da-e4b02e724aaa" (UID: "d6706295-66df-48cf-a8da-e4b02e724aaa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:27:30.620589 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:30.620569    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6706295-66df-48cf-a8da-e4b02e724aaa-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:30.622243 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:30.622220    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6706295-66df-48cf-a8da-e4b02e724aaa-metrics-certs" pod "d6706295-66df-48cf-a8da-e4b02e724aaa" (UID: "d6706295-66df-48cf-a8da-e4b02e724aaa"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:27:30.622335 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:30.622242    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6706295-66df-48cf-a8da-e4b02e724aaa-maas-api-tls" pod "d6706295-66df-48cf-a8da-e4b02e724aaa" (UID: "d6706295-66df-48cf-a8da-e4b02e724aaa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:27:30.622335 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:30.622277    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6706295-66df-48cf-a8da-e4b02e724aaa-kube-api-access-t7nmc" pod "d6706295-66df-48cf-a8da-e4b02e724aaa" (UID: "d6706295-66df-48cf-a8da-e4b02e724aaa"). InnerVolumeSpecName "kube-api-access-t7nmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:30.742068 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:30.742031    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6706295-66df-48cf-a8da-e4b02e724aaa-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:30.742311 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:30.742287    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7nmc\" (UniqueName: \"kubernetes.io/projected/d6706295-66df-48cf-a8da-e4b02e724aaa-kube-api-access-t7nmc\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:30.742311 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:30.742310    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6706295-66df-48cf-a8da-e4b02e724aaa-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:31.484680 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:31.484648    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"
Oct 30 19:27:31.484903 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:31.484686    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c" event={"ID":"d6706295-66df-48cf-a8da-e4b02e724aaa","Type":"ContainerDied","Data":"78bd9f94d414668976b19f5d0a85de6414897204d1be46277123b99191581821"}
Oct 30 19:27:31.484903 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:31.484743    5820 scope.go:122] "RemoveContainer" containerID="aeebc5bea6eb43616c6f0ece1e8c2a97d547e1b10754ad23a39c442afb515bea"
Oct 30 19:27:31.657503 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:31.657463    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"]
Oct 30 19:27:31.704720 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:31.704660    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-c60f5c-86f8bd5-v4z4c"]
Oct 30 19:27:32.363489 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:32.363456    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:33.436915 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:33.436881    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d6706295-66df-48cf-a8da-e4b02e724aaa" path="/var/lib/kubelet/pods/d6706295-66df-48cf-a8da-e4b02e724aaa/volumes"
Oct 30 19:27:34.017343 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:34.017308    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:34.300784 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:34.300750    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:35.083845 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:35.083808    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:37.692013 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:37.691978    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:38.203309 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:27:38.203275    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:27:41.536353 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:41.536316    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:41.758087 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:41.758053    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:41.966211 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:41.966149    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:42.585214 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:42.585166    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:42.834389 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:42.834334    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:43.306494 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:43.306459    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:43.545061 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:43.545029    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:47.810207 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:47.810168    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:48.032135 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:48.032094    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:48.264779 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:48.264701    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:51.339473 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:51.339426    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:51.568565 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:51.568529    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:51.724523 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:51.724442    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf"]
Oct 30 19:27:51.725166 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:51.725147    5820 cpu_manager.go:414] "containerMap: removing container" podUID="9dbe29c1-0a26-4859-a57c-460ddc518f24" containerName="payload-processing"
Oct 30 19:27:51.725245 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:51.725167    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9dbe29c1-0a26-4859-a57c-460ddc518f24" containerName="payload-processing"
Oct 30 19:27:51.725245 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:51.725179    5820 cpu_manager.go:414] "containerMap: removing container" podUID="d6706295-66df-48cf-a8da-e4b02e724aaa" containerName="maas-api"
Oct 30 19:27:51.725245 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:51.725184    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6706295-66df-48cf-a8da-e4b02e724aaa" containerName="maas-api"
Oct 30 19:27:51.725245 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:51.725194    5820 cpu_manager.go:414] "containerMap: removing container" podUID="dba918a1-39a1-41ad-bb05-55a5c5f6981c" containerName="payload-pre-processing"
Oct 30 19:27:51.725245 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:51.725200    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dba918a1-39a1-41ad-bb05-55a5c5f6981c" containerName="payload-pre-processing"
Oct 30 19:27:51.725411 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:51.725321    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="9dbe29c1-0a26-4859-a57c-460ddc518f24" containerName="payload-processing"
Oct 30 19:27:51.725411 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:51.725330    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6706295-66df-48cf-a8da-e4b02e724aaa" containerName="maas-api"
Oct 30 19:27:51.725411 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:51.725338    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="dba918a1-39a1-41ad-bb05-55a5c5f6981c" containerName="payload-pre-processing"
Oct 30 19:27:51.730100 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:51.730039    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf"
Oct 30 19:27:51.785853 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:51.785828    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf"]
Oct 30 19:27:51.794766 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:51.794735    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98046a89-b091-4e08-9696-985da1473d2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf\" (UID: \"98046a89-b091-4e08-9696-985da1473d2a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf"
Oct 30 19:27:51.794943 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:51.794919    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgl5g\" (UniqueName: \"kubernetes.io/projected/98046a89-b091-4e08-9696-985da1473d2a-kube-api-access-qgl5g\") pod \"payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf\" (UID: \"98046a89-b091-4e08-9696-985da1473d2a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf"
Oct 30 19:27:51.832119 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:51.832091    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-200c46-dockercfg-mhk4v\""
Oct 30 19:27:51.832414 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:51.832393    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-200c46\""
Oct 30 19:27:51.896135 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:51.896097    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98046a89-b091-4e08-9696-985da1473d2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf\" (UID: \"98046a89-b091-4e08-9696-985da1473d2a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf"
Oct 30 19:27:51.896348 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:51.896195    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgl5g\" (UniqueName: \"kubernetes.io/projected/98046a89-b091-4e08-9696-985da1473d2a-kube-api-access-qgl5g\") pod \"payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf\" (UID: \"98046a89-b091-4e08-9696-985da1473d2a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf"
Oct 30 19:27:51.896880 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:51.896852    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98046a89-b091-4e08-9696-985da1473d2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf\" (UID: \"98046a89-b091-4e08-9696-985da1473d2a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf"
Oct 30 19:27:51.957782 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:51.957749    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgl5g\" (UniqueName: \"kubernetes.io/projected/98046a89-b091-4e08-9696-985da1473d2a-kube-api-access-qgl5g\") pod \"payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf\" (UID: \"98046a89-b091-4e08-9696-985da1473d2a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf"
Oct 30 19:27:52.043762 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:52.043693    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf"
Oct 30 19:27:52.423921 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:52.423896    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf"]
Oct 30 19:27:52.425467 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:27:52.425441    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98046a89_b091_4e08_9696_985da1473d2a.slice/crio-facda04052cbd8706b73254528bffe2cd71fac5894079aa6b95cce3f3fdf0463 WatchSource:0}: Error finding container facda04052cbd8706b73254528bffe2cd71fac5894079aa6b95cce3f3fdf0463: Status 404 returned error can't find the container with id facda04052cbd8706b73254528bffe2cd71fac5894079aa6b95cce3f3fdf0463
Oct 30 19:27:52.622723 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:52.622683    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf" event={"ID":"98046a89-b091-4e08-9696-985da1473d2a","Type":"ContainerStarted","Data":"aaf14a7e190c6d0ed2f58b6cc337db08cd7eea9fcb9dcaf1cdaa52e2e6ad4a20"}
Oct 30 19:27:52.622885 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:52.622746    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf" event={"ID":"98046a89-b091-4e08-9696-985da1473d2a","Type":"ContainerStarted","Data":"facda04052cbd8706b73254528bffe2cd71fac5894079aa6b95cce3f3fdf0463"}
Oct 30 19:27:52.622885 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:52.622773    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf"
Oct 30 19:27:52.739918 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:52.739825    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf" podStartSLOduration=1.739809627 podStartE2EDuration="1.739809627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:52.73818319 +0000 UTC m=+3193.449931887" watchObservedRunningTime="2026-10-30 19:27:52.739809627 +0000 UTC m=+3193.451558368"
Oct 30 19:27:53.045596 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:53.045510    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:57.834396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:27:57.834352    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:02.697357 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:02.697310    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:03.632037 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:03.632009    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf"
Oct 30 19:28:04.670444 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:04.670409    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:07.562985 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:07.562923    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:08.204168 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:28:08.204134    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:28:12.478643 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:12.478510    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:17.441617 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:17.441581    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:23.385098 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:23.385056    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:24.014625 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:24.014588    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:27.127172 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:27.127133    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:31.933445 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:31.933404    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:36.739912 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:36.739871    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:38.203755 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:28:38.203720    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:28:41.575067 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:41.575027    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:46.325827 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:46.325785    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:50.763307 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:50.763220    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"]
Oct 30 19:28:50.767465 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:50.767449    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"
Oct 30 19:28:50.819601 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:50.819572    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-200c46-serving-cert\""
Oct 30 19:28:50.856043 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:50.856007    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"]
Oct 30 19:28:50.928307 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:50.928281    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6038d170-874b-48fc-8665-982bff9cdc30-maas-api-tls\") pod \"maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8\" (UID: \"6038d170-874b-48fc-8665-982bff9cdc30\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"
Oct 30 19:28:50.928506 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:50.928484    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6038d170-874b-48fc-8665-982bff9cdc30-openshift-service-ca\") pod \"maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8\" (UID: \"6038d170-874b-48fc-8665-982bff9cdc30\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"
Oct 30 19:28:50.928553 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:50.928526    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6038d170-874b-48fc-8665-982bff9cdc30-metrics-certs\") pod \"maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8\" (UID: \"6038d170-874b-48fc-8665-982bff9cdc30\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"
Oct 30 19:28:50.928602 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:50.928591    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48bkm\" (UniqueName: \"kubernetes.io/projected/6038d170-874b-48fc-8665-982bff9cdc30-kube-api-access-48bkm\") pod \"maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8\" (UID: \"6038d170-874b-48fc-8665-982bff9cdc30\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"
Oct 30 19:28:50.988650 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:50.988615    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:51.030084 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:51.029992    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6038d170-874b-48fc-8665-982bff9cdc30-openshift-service-ca\") pod \"maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8\" (UID: \"6038d170-874b-48fc-8665-982bff9cdc30\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"
Oct 30 19:28:51.030261 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:51.030081    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6038d170-874b-48fc-8665-982bff9cdc30-metrics-certs\") pod \"maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8\" (UID: \"6038d170-874b-48fc-8665-982bff9cdc30\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"
Oct 30 19:28:51.030261 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:51.030153    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48bkm\" (UniqueName: \"kubernetes.io/projected/6038d170-874b-48fc-8665-982bff9cdc30-kube-api-access-48bkm\") pod \"maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8\" (UID: \"6038d170-874b-48fc-8665-982bff9cdc30\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"
Oct 30 19:28:51.030399 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:51.030262    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6038d170-874b-48fc-8665-982bff9cdc30-maas-api-tls\") pod \"maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8\" (UID: \"6038d170-874b-48fc-8665-982bff9cdc30\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"
Oct 30 19:28:51.030707 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:51.030684    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6038d170-874b-48fc-8665-982bff9cdc30-openshift-service-ca\") pod \"maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8\" (UID: \"6038d170-874b-48fc-8665-982bff9cdc30\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"
Oct 30 19:28:51.032892 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:51.032858    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6038d170-874b-48fc-8665-982bff9cdc30-maas-api-tls\") pod \"maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8\" (UID: \"6038d170-874b-48fc-8665-982bff9cdc30\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"
Oct 30 19:28:51.033023 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:51.033010    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6038d170-874b-48fc-8665-982bff9cdc30-metrics-certs\") pod \"maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8\" (UID: \"6038d170-874b-48fc-8665-982bff9cdc30\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"
Oct 30 19:28:51.094324 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:51.094300    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48bkm\" (UniqueName: \"kubernetes.io/projected/6038d170-874b-48fc-8665-982bff9cdc30-kube-api-access-48bkm\") pod \"maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8\" (UID: \"6038d170-874b-48fc-8665-982bff9cdc30\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"
Oct 30 19:28:51.221418 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:51.221383    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:51.383228 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:51.383195    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"
Oct 30 19:28:51.772514 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:51.772484    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"]
Oct 30 19:28:51.772901 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:28:51.772871    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6038d170_874b_48fc_8665_982bff9cdc30.slice/crio-14b4e7350da86ccff4d6c067e9e27f7a3b0ff8e4c097ff8a0a0e5d222364bf0b WatchSource:0}: Error finding container 14b4e7350da86ccff4d6c067e9e27f7a3b0ff8e4c097ff8a0a0e5d222364bf0b: Status 404 returned error can't find the container with id 14b4e7350da86ccff4d6c067e9e27f7a3b0ff8e4c097ff8a0a0e5d222364bf0b
Oct 30 19:28:51.775543 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:51.775526    5820 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:28:51.997101 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:51.997072    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8" event={"ID":"6038d170-874b-48fc-8665-982bff9cdc30","Type":"ContainerStarted","Data":"14b4e7350da86ccff4d6c067e9e27f7a3b0ff8e4c097ff8a0a0e5d222364bf0b"}
Oct 30 19:28:53.005390 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:53.005356    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8" event={"ID":"6038d170-874b-48fc-8665-982bff9cdc30","Type":"ContainerStarted","Data":"4068a283c19eea38f4a2a83ee9c3266d5dee8fc220c4d8260d9e60bc1cc09910"}
Oct 30 19:28:53.005778 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:53.005509    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"
Oct 30 19:28:53.123462 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:53.123412    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8" podStartSLOduration=2.1138338660000002 podStartE2EDuration="3.123397957s" totalImagesPullingTime="1.009564091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:53.122419785 +0000 UTC m=+3253.834168481" watchObservedRunningTime="2026-10-30 19:28:53.123397957 +0000 UTC m=+3253.835146653"
Oct 30 19:28:57.310737 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:57.310695    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:57.520513 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:57.520474    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:57.859784 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:57.859749    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:59.017360 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:28:59.017330    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"
Oct 30 19:29:02.211171 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:02.211040    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:02.776863 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:02.776829    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"]
Oct 30 19:29:02.981626 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:02.981580    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:05.021830 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:05.021780    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:07.042463 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:07.042423    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:08.203872 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:29:08.203835    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:29:11.991600 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:11.991560    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:16.960612 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:16.960570    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:17.798477 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:17.798427    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg" podUID="354bfaa6-d937-437c-bcd9-48a7db460d23" containerName="main" containerID="cri-o://c37ecdd6b8dacb56d871eb421c409c9612eb8079d8117b77cfe3057f29027264" gracePeriod=45
Oct 30 19:29:18.068731 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.068710    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:29:18.146892 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.146862    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-dshm\" (UniqueName: \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-dshm\") pod \"354bfaa6-d937-437c-bcd9-48a7db460d23\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") "
Oct 30 19:29:18.147046 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.146923    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-home\" (UniqueName: \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-home\") pod \"354bfaa6-d937-437c-bcd9-48a7db460d23\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") "
Oct 30 19:29:18.147046 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.147007    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/354bfaa6-d937-437c-bcd9-48a7db460d23-kube-api-access-v6lhr\" (UniqueName: \"kubernetes.io/projected/354bfaa6-d937-437c-bcd9-48a7db460d23-kube-api-access-v6lhr\") pod \"354bfaa6-d937-437c-bcd9-48a7db460d23\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") "
Oct 30 19:29:18.147239 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.147063    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/354bfaa6-d937-437c-bcd9-48a7db460d23-tls-certs\" (UniqueName: \"kubernetes.io/secret/354bfaa6-d937-437c-bcd9-48a7db460d23-tls-certs\") pod \"354bfaa6-d937-437c-bcd9-48a7db460d23\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") "
Oct 30 19:29:18.147239 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.147093    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-tmp-dir\") pod \"354bfaa6-d937-437c-bcd9-48a7db460d23\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") "
Oct 30 19:29:18.147239 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.147135    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-home" pod "354bfaa6-d937-437c-bcd9-48a7db460d23" (UID: "354bfaa6-d937-437c-bcd9-48a7db460d23"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:29:18.147239 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.147192    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-model-cache\") pod \"354bfaa6-d937-437c-bcd9-48a7db460d23\" (UID: \"354bfaa6-d937-437c-bcd9-48a7db460d23\") "
Oct 30 19:29:18.147457 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.147369    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-tmp-dir" pod "354bfaa6-d937-437c-bcd9-48a7db460d23" (UID: "354bfaa6-d937-437c-bcd9-48a7db460d23"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:29:18.147499 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.147457    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-model-cache" pod "354bfaa6-d937-437c-bcd9-48a7db460d23" (UID: "354bfaa6-d937-437c-bcd9-48a7db460d23"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:29:18.147690 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.147667    5820 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-tmp-dir\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:18.147786 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.147690    5820 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-model-cache\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:18.147786 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.147703    5820 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-home\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:18.149314 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.149296    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-dshm" pod "354bfaa6-d937-437c-bcd9-48a7db460d23" (UID: "354bfaa6-d937-437c-bcd9-48a7db460d23"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:29:18.149854 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.149836    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/354bfaa6-d937-437c-bcd9-48a7db460d23-tls-certs" pod "354bfaa6-d937-437c-bcd9-48a7db460d23" (UID: "354bfaa6-d937-437c-bcd9-48a7db460d23"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:29:18.149935 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.149865    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/354bfaa6-d937-437c-bcd9-48a7db460d23-kube-api-access-v6lhr" pod "354bfaa6-d937-437c-bcd9-48a7db460d23" (UID: "354bfaa6-d937-437c-bcd9-48a7db460d23"). InnerVolumeSpecName "kube-api-access-v6lhr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:29:18.166577 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.166548    5820 generic.go:356] "Generic (PLEG): container finished" podID="354bfaa6-d937-437c-bcd9-48a7db460d23" containerID="c37ecdd6b8dacb56d871eb421c409c9612eb8079d8117b77cfe3057f29027264" exitCode=0
Oct 30 19:29:18.166666 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.166585    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"
Oct 30 19:29:18.166666 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.166620    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg" event={"ID":"354bfaa6-d937-437c-bcd9-48a7db460d23","Type":"ContainerDied","Data":"c37ecdd6b8dacb56d871eb421c409c9612eb8079d8117b77cfe3057f29027264"}
Oct 30 19:29:18.166666 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.166660    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg" event={"ID":"354bfaa6-d937-437c-bcd9-48a7db460d23","Type":"ContainerDied","Data":"d9b274b7bb737a9dca7eb1330553ef444565622487c34821c761c5b6598b1e7c"}
Oct 30 19:29:18.166789 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.166676    5820 scope.go:122] "RemoveContainer" containerID="c37ecdd6b8dacb56d871eb421c409c9612eb8079d8117b77cfe3057f29027264"
Oct 30 19:29:18.177022 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.177003    5820 scope.go:122] "RemoveContainer" containerID="c37ecdd6b8dacb56d871eb421c409c9612eb8079d8117b77cfe3057f29027264"
Oct 30 19:29:18.177285 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:29:18.177267    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c37ecdd6b8dacb56d871eb421c409c9612eb8079d8117b77cfe3057f29027264\": container with ID starting with c37ecdd6b8dacb56d871eb421c409c9612eb8079d8117b77cfe3057f29027264 not found: ID does not exist" containerID="c37ecdd6b8dacb56d871eb421c409c9612eb8079d8117b77cfe3057f29027264"
Oct 30 19:29:18.177341 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.177293    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c37ecdd6b8dacb56d871eb421c409c9612eb8079d8117b77cfe3057f29027264"} err="failed to get container status \"c37ecdd6b8dacb56d871eb421c409c9612eb8079d8117b77cfe3057f29027264\": rpc error: code = NotFound desc = could not find container \"c37ecdd6b8dacb56d871eb421c409c9612eb8079d8117b77cfe3057f29027264\": container with ID starting with c37ecdd6b8dacb56d871eb421c409c9612eb8079d8117b77cfe3057f29027264 not found: ID does not exist"
Oct 30 19:29:18.249430 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.249399    5820 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/354bfaa6-d937-437c-bcd9-48a7db460d23-dshm\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:18.249430 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.249431    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6lhr\" (UniqueName: \"kubernetes.io/projected/354bfaa6-d937-437c-bcd9-48a7db460d23-kube-api-access-v6lhr\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:18.249639 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.249446    5820 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/354bfaa6-d937-437c-bcd9-48a7db460d23-tls-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:18.300898 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.300822    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"]
Oct 30 19:29:18.304293 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:18.304266    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-praxis-a-81d66e/praxis-route-81d66e-kserve-66bccdcc6f-kw4lg"]
Oct 30 19:29:19.437963 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:19.437911    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="354bfaa6-d937-437c-bcd9-48a7db460d23" path="/var/lib/kubelet/pods/354bfaa6-d937-437c-bcd9-48a7db460d23/volumes"
Oct 30 19:29:21.786727 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:21.786689    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:22.014700 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:22.014660    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:23.310229 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:23.310183    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:26.789199 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:26.789163    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:29.616119 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:29.616078    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:29.850851 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:29.850820    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:29.906211 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:29.906139    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"]
Oct 30 19:29:29.906402 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:29.906367    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp" podUID="c86c20cc-52fc-4ffb-aa87-46fcc272c457" containerName="istio-proxy" containerID="cri-o://2387be425d06d330d7e35f16350aec3452ad362db57738bfca7c11700e5a8be5" gracePeriod=30
Oct 30 19:29:30.049595 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:30.049559    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:31.945704 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:31.945671    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:32.305846 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:32.305811    5820 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:29:32.306067 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:32.305875    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp" podUID="c86c20cc-52fc-4ffb-aa87-46fcc272c457" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:29:35.177934 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.177862    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:29:35.234607 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.234574    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-credential-socket\") pod \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") "
Oct 30 19:29:35.234778 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.234639    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c86c20cc-52fc-4ffb-aa87-46fcc272c457-kube-api-access-ddw85\" (UniqueName: \"kubernetes.io/projected/c86c20cc-52fc-4ffb-aa87-46fcc272c457-kube-api-access-ddw85\") pod \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") "
Oct 30 19:29:35.234778 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.234667    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-token\" (UniqueName: \"kubernetes.io/projected/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-token\") pod \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") "
Oct 30 19:29:35.234911 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.234830    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-workload-socket\") pod \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") "
Oct 30 19:29:35.234911 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.234860    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-credential-socket" pod "c86c20cc-52fc-4ffb-aa87-46fcc272c457" (UID: "c86c20cc-52fc-4ffb-aa87-46fcc272c457"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:29:35.234911 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.234898    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-podinfo\") pod \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") "
Oct 30 19:29:35.235076 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.234996    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-envoy\") pod \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") "
Oct 30 19:29:35.235076 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.235030    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istiod-ca-cert\") pod \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") "
Oct 30 19:29:35.235076 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.235068    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-data\") pod \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") "
Oct 30 19:29:35.235252 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.235111    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-workload-socket" pod "c86c20cc-52fc-4ffb-aa87-46fcc272c457" (UID: "c86c20cc-52fc-4ffb-aa87-46fcc272c457"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:29:35.235252 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.235133    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-workload-certs\") pod \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\" (UID: \"c86c20cc-52fc-4ffb-aa87-46fcc272c457\") "
Oct 30 19:29:35.235373 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.235323    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-data" pod "c86c20cc-52fc-4ffb-aa87-46fcc272c457" (UID: "c86c20cc-52fc-4ffb-aa87-46fcc272c457"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:29:35.235472 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.235450    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istiod-ca-cert" pod "c86c20cc-52fc-4ffb-aa87-46fcc272c457" (UID: "c86c20cc-52fc-4ffb-aa87-46fcc272c457"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:29:35.235715 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.235695    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-workload-certs" pod "c86c20cc-52fc-4ffb-aa87-46fcc272c457" (UID: "c86c20cc-52fc-4ffb-aa87-46fcc272c457"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:29:35.235826 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.235804    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:35.235900 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.235833    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:35.235900 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.235849    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:35.235900 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.235863    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:35.235900 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.235877    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:35.237437 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.237407    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-token" pod "c86c20cc-52fc-4ffb-aa87-46fcc272c457" (UID: "c86c20cc-52fc-4ffb-aa87-46fcc272c457"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:29:35.237663 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.237637    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c86c20cc-52fc-4ffb-aa87-46fcc272c457-kube-api-access-ddw85" pod "c86c20cc-52fc-4ffb-aa87-46fcc272c457" (UID: "c86c20cc-52fc-4ffb-aa87-46fcc272c457"). InnerVolumeSpecName "kube-api-access-ddw85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:29:35.237739 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.237643    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-podinfo" pod "c86c20cc-52fc-4ffb-aa87-46fcc272c457" (UID: "c86c20cc-52fc-4ffb-aa87-46fcc272c457"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:29:35.237993 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.237932    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-envoy" pod "c86c20cc-52fc-4ffb-aa87-46fcc272c457" (UID: "c86c20cc-52fc-4ffb-aa87-46fcc272c457"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:29:35.275757 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.275728    5820 generic.go:356] "Generic (PLEG): container finished" podID="c86c20cc-52fc-4ffb-aa87-46fcc272c457" containerID="2387be425d06d330d7e35f16350aec3452ad362db57738bfca7c11700e5a8be5" exitCode=0
Oct 30 19:29:35.275860 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.275771    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp" event={"ID":"c86c20cc-52fc-4ffb-aa87-46fcc272c457","Type":"ContainerDied","Data":"2387be425d06d330d7e35f16350aec3452ad362db57738bfca7c11700e5a8be5"}
Oct 30 19:29:35.275860 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.275812    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp" event={"ID":"c86c20cc-52fc-4ffb-aa87-46fcc272c457","Type":"ContainerDied","Data":"533ab8932b7b13081936d4f3c7d6f50890181759df79775a87ec0a661bcb516a"}
Oct 30 19:29:35.275860 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.275835    5820 scope.go:122] "RemoveContainer" containerID="2387be425d06d330d7e35f16350aec3452ad362db57738bfca7c11700e5a8be5"
Oct 30 19:29:35.275860 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.275773    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"
Oct 30 19:29:35.286039 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.286023    5820 scope.go:122] "RemoveContainer" containerID="2387be425d06d330d7e35f16350aec3452ad362db57738bfca7c11700e5a8be5"
Oct 30 19:29:35.286305 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:29:35.286287    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2387be425d06d330d7e35f16350aec3452ad362db57738bfca7c11700e5a8be5\": container with ID starting with 2387be425d06d330d7e35f16350aec3452ad362db57738bfca7c11700e5a8be5 not found: ID does not exist" containerID="2387be425d06d330d7e35f16350aec3452ad362db57738bfca7c11700e5a8be5"
Oct 30 19:29:35.286362 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.286316    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2387be425d06d330d7e35f16350aec3452ad362db57738bfca7c11700e5a8be5"} err="failed to get container status \"2387be425d06d330d7e35f16350aec3452ad362db57738bfca7c11700e5a8be5\": rpc error: code = NotFound desc = could not find container \"2387be425d06d330d7e35f16350aec3452ad362db57738bfca7c11700e5a8be5\": container with ID starting with 2387be425d06d330d7e35f16350aec3452ad362db57738bfca7c11700e5a8be5 not found: ID does not exist"
Oct 30 19:29:35.336525 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.336501    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddw85\" (UniqueName: \"kubernetes.io/projected/c86c20cc-52fc-4ffb-aa87-46fcc272c457-kube-api-access-ddw85\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:35.336525 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.336523    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:35.336664 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.336532    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:35.336664 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.336541    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c86c20cc-52fc-4ffb-aa87-46fcc272c457-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:35.461504 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.461432    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"]
Oct 30 19:29:35.467931 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:35.467895    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-mfxsp"]
Oct 30 19:29:36.834637 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:36.834595    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:37.436791 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:37.436758    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c86c20cc-52fc-4ffb-aa87-46fcc272c457" path="/var/lib/kubelet/pods/c86c20cc-52fc-4ffb-aa87-46fcc272c457/volumes"
Oct 30 19:29:38.203448 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:29:38.203403    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:29:41.832259 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:41.832219    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:42.735825 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:42.735789    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:42.916140 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:42.916102    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:46.639936 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:46.639831    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:51.465993 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:51.465932    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:56.372440 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:29:56.372401    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:01.312408 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:01.312277    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:05.382146 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:05.382110    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:06.165777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:06.165606    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:08.203665 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:30:08.203623    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:30:11.532413 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:11.532358    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:11.792335 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:11.792245    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:15.194809 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:15.194769    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:16.343573 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:16.343535    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:20.398665 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:20.398584    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"]
Oct 30 19:30:20.399065 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:20.398841    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp" podUID="a84845b1-4310-4b17-949d-e4ee705f240e" containerName="maas-api" containerID="cri-o://fd5e152102f198484d7bb011318c4dfe9af393e62bb89fbb766a52a567282ffe" gracePeriod=30
Oct 30 19:30:20.539077 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:20.539043    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:20.569360 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:20.569323    5820 generic.go:356] "Generic (PLEG): container finished" podID="a84845b1-4310-4b17-949d-e4ee705f240e" containerID="fd5e152102f198484d7bb011318c4dfe9af393e62bb89fbb766a52a567282ffe" exitCode=0
Oct 30 19:30:20.569542 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:20.569372    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp" event={"ID":"a84845b1-4310-4b17-949d-e4ee705f240e","Type":"ContainerDied","Data":"fd5e152102f198484d7bb011318c4dfe9af393e62bb89fbb766a52a567282ffe"}
Oct 30 19:30:20.703828 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:20.703799    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"
Oct 30 19:30:20.869137 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:20.869099    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a84845b1-4310-4b17-949d-e4ee705f240e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a84845b1-4310-4b17-949d-e4ee705f240e-openshift-service-ca\") pod \"a84845b1-4310-4b17-949d-e4ee705f240e\" (UID: \"a84845b1-4310-4b17-949d-e4ee705f240e\") "
Oct 30 19:30:20.869356 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:20.869319    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a84845b1-4310-4b17-949d-e4ee705f240e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a84845b1-4310-4b17-949d-e4ee705f240e-metrics-certs\") pod \"a84845b1-4310-4b17-949d-e4ee705f240e\" (UID: \"a84845b1-4310-4b17-949d-e4ee705f240e\") "
Oct 30 19:30:20.869497 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:20.869372    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a84845b1-4310-4b17-949d-e4ee705f240e-kube-api-access-28cp8\" (UniqueName: \"kubernetes.io/projected/a84845b1-4310-4b17-949d-e4ee705f240e-kube-api-access-28cp8\") pod \"a84845b1-4310-4b17-949d-e4ee705f240e\" (UID: \"a84845b1-4310-4b17-949d-e4ee705f240e\") "
Oct 30 19:30:20.869497 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:20.869406    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a84845b1-4310-4b17-949d-e4ee705f240e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a84845b1-4310-4b17-949d-e4ee705f240e-maas-api-tls\") pod \"a84845b1-4310-4b17-949d-e4ee705f240e\" (UID: \"a84845b1-4310-4b17-949d-e4ee705f240e\") "
Oct 30 19:30:20.869497 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:20.869451    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a84845b1-4310-4b17-949d-e4ee705f240e-openshift-service-ca" pod "a84845b1-4310-4b17-949d-e4ee705f240e" (UID: "a84845b1-4310-4b17-949d-e4ee705f240e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:20.869916 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:20.869891    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a84845b1-4310-4b17-949d-e4ee705f240e-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:20.871782 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:20.871761    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a84845b1-4310-4b17-949d-e4ee705f240e-metrics-certs" pod "a84845b1-4310-4b17-949d-e4ee705f240e" (UID: "a84845b1-4310-4b17-949d-e4ee705f240e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:30:20.871888 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:20.871783    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a84845b1-4310-4b17-949d-e4ee705f240e-maas-api-tls" pod "a84845b1-4310-4b17-949d-e4ee705f240e" (UID: "a84845b1-4310-4b17-949d-e4ee705f240e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:30:20.871929 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:20.871896    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a84845b1-4310-4b17-949d-e4ee705f240e-kube-api-access-28cp8" pod "a84845b1-4310-4b17-949d-e4ee705f240e" (UID: "a84845b1-4310-4b17-949d-e4ee705f240e"). InnerVolumeSpecName "kube-api-access-28cp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:20.970983 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:20.970882    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a84845b1-4310-4b17-949d-e4ee705f240e-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:20.970983 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:20.970919    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28cp8\" (UniqueName: \"kubernetes.io/projected/a84845b1-4310-4b17-949d-e4ee705f240e-kube-api-access-28cp8\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:20.970983 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:20.970930    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a84845b1-4310-4b17-949d-e4ee705f240e-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:21.576796 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:21.576755    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp" event={"ID":"a84845b1-4310-4b17-949d-e4ee705f240e","Type":"ContainerDied","Data":"581ce0e7fb5bb4adf92b0acf45bc2d39fe66df06bd8ee7436f7c5df22fbf3a19"}
Oct 30 19:30:21.577274 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:21.576814    5820 scope.go:122] "RemoveContainer" containerID="fd5e152102f198484d7bb011318c4dfe9af393e62bb89fbb766a52a567282ffe"
Oct 30 19:30:21.577274 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:21.576770    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"
Oct 30 19:30:21.745064 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:21.744981    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"]
Oct 30 19:30:21.794235 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:21.794200    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-f5dfb8-86b45f5b84-5dlqp"]
Oct 30 19:30:22.032979 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:22.032880    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:22.251694 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:22.251658    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:22.459721 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:22.459689    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:23.062676 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:23.062638    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:23.438478 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:23.438403    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a84845b1-4310-4b17-949d-e4ee705f240e" path="/var/lib/kubelet/pods/a84845b1-4310-4b17-949d-e4ee705f240e/volumes"
Oct 30 19:30:27.421417 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:27.421378    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:29.826365 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:29.826325    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:30.034282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:30.034252    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:30.218585 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:30.218475    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:31.045739 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:31.045700    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:31.978670 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:31.978590    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:32.262739 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:32.262652    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:32.456327 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:32.456293    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:33.630001 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:33.629960    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:36.416817 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:36.416770    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:36.643415 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:36.643370    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:37.416307 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:37.416269    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:38.204406 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:30:38.204361    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:30:38.291848 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:38.291817    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:38.917944 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:38.917908    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:39.166072 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:39.166031    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:42.054334 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:42.054296    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:42.283219 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:42.283178    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:42.692207 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:42.692171    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:43.579780 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:43.579739    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:43.762599 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:43.762561    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:43.964897 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:43.964779    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:47.035009 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.034969    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:47.345262 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.345230    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:47.587042 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.587009    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq"]
Oct 30 19:30:47.587721 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.587701    5820 cpu_manager.go:414] "containerMap: removing container" podUID="c86c20cc-52fc-4ffb-aa87-46fcc272c457" containerName="istio-proxy"
Oct 30 19:30:47.587777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.587724    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c86c20cc-52fc-4ffb-aa87-46fcc272c457" containerName="istio-proxy"
Oct 30 19:30:47.587777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.587735    5820 cpu_manager.go:414] "containerMap: removing container" podUID="354bfaa6-d937-437c-bcd9-48a7db460d23" containerName="main"
Oct 30 19:30:47.587777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.587741    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="354bfaa6-d937-437c-bcd9-48a7db460d23" containerName="main"
Oct 30 19:30:47.587777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.587757    5820 cpu_manager.go:414] "containerMap: removing container" podUID="a84845b1-4310-4b17-949d-e4ee705f240e" containerName="maas-api"
Oct 30 19:30:47.587777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.587762    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a84845b1-4310-4b17-949d-e4ee705f240e" containerName="maas-api"
Oct 30 19:30:47.587974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.587866    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="c86c20cc-52fc-4ffb-aa87-46fcc272c457" containerName="istio-proxy"
Oct 30 19:30:47.587974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.587875    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="354bfaa6-d937-437c-bcd9-48a7db460d23" containerName="main"
Oct 30 19:30:47.587974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.587882    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="a84845b1-4310-4b17-949d-e4ee705f240e" containerName="maas-api"
Oct 30 19:30:47.592326 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.592305    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq"
Oct 30 19:30:47.645815 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.645786    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-7fdf6b\""
Oct 30 19:30:47.646224 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.646200    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-7fdf6b-dockercfg-ftnvv\""
Oct 30 19:30:47.649125 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.649093    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq"]
Oct 30 19:30:47.710341 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.710313    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a6148bf-297b-4a6a-aa29-bd4d769a4d5c-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq\" (UID: \"0a6148bf-297b-4a6a-aa29-bd4d769a4d5c\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq"
Oct 30 19:30:47.710499 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.710377    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss47b\" (UniqueName: \"kubernetes.io/projected/0a6148bf-297b-4a6a-aa29-bd4d769a4d5c-kube-api-access-ss47b\") pod \"payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq\" (UID: \"0a6148bf-297b-4a6a-aa29-bd4d769a4d5c\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq"
Oct 30 19:30:47.811483 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.811448    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a6148bf-297b-4a6a-aa29-bd4d769a4d5c-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq\" (UID: \"0a6148bf-297b-4a6a-aa29-bd4d769a4d5c\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq"
Oct 30 19:30:47.811652 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.811514    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss47b\" (UniqueName: \"kubernetes.io/projected/0a6148bf-297b-4a6a-aa29-bd4d769a4d5c-kube-api-access-ss47b\") pod \"payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq\" (UID: \"0a6148bf-297b-4a6a-aa29-bd4d769a4d5c\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq"
Oct 30 19:30:47.812137 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.812119    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a6148bf-297b-4a6a-aa29-bd4d769a4d5c-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq\" (UID: \"0a6148bf-297b-4a6a-aa29-bd4d769a4d5c\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq"
Oct 30 19:30:47.872669 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.872608    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss47b\" (UniqueName: \"kubernetes.io/projected/0a6148bf-297b-4a6a-aa29-bd4d769a4d5c-kube-api-access-ss47b\") pod \"payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq\" (UID: \"0a6148bf-297b-4a6a-aa29-bd4d769a4d5c\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq"
Oct 30 19:30:47.906777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.906753    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq"
Oct 30 19:30:47.981197 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:47.980903    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:48.285841 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:48.285810    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq"]
Oct 30 19:30:48.286902 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:30:48.286857    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a6148bf_297b_4a6a_aa29_bd4d769a4d5c.slice/crio-e95345f1c333ebedab3c9b12feb47fbb7cfc4be8350b4f5682b61a93946046c3 WatchSource:0}: Error finding container e95345f1c333ebedab3c9b12feb47fbb7cfc4be8350b4f5682b61a93946046c3: Status 404 returned error can't find the container with id e95345f1c333ebedab3c9b12feb47fbb7cfc4be8350b4f5682b61a93946046c3
Oct 30 19:30:48.742729 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:48.742696    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq" event={"ID":"0a6148bf-297b-4a6a-aa29-bd4d769a4d5c","Type":"ContainerStarted","Data":"fc8d6f7b72324197f2402a8215c10c84be6d9f2305800d8903c30b36e0b64c35"}
Oct 30 19:30:48.744800 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:48.742746    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq" event={"ID":"0a6148bf-297b-4a6a-aa29-bd4d769a4d5c","Type":"ContainerStarted","Data":"e95345f1c333ebedab3c9b12feb47fbb7cfc4be8350b4f5682b61a93946046c3"}
Oct 30 19:30:48.744800 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:48.742793    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq"
Oct 30 19:30:48.859736 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:48.859675    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq" podStartSLOduration=1.85965612 podStartE2EDuration="1.85965612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:48.858513004 +0000 UTC m=+3369.570261701" watchObservedRunningTime="2026-10-30 19:30:48.85965612 +0000 UTC m=+3369.571404818"
Oct 30 19:30:52.818379 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:52.818050    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:57.560571 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:57.560533    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:59.752526 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:30:59.752497    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq"
Oct 30 19:31:02.472436 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:02.472396    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:05.728035 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:05.727996    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:07.331996 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:07.331526    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:08.204094 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:31:08.204055    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:31:12.112547 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:12.112509    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:17.177290 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:17.177232    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:22.624834 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:22.624797    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:26.829797 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:26.829761    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf"]
Oct 30 19:31:27.158528 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:27.158365    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:27.361488 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:27.361446    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:27.570808 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:27.570774    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:31.830478 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:31.830437    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf" podUID="98046a89-b091-4e08-9696-985da1473d2a" containerName="payload-pre-processing" containerID="cri-o://aaf14a7e190c6d0ed2f58b6cc337db08cd7eea9fcb9dcaf1cdaa52e2e6ad4a20" gracePeriod=25
Oct 30 19:31:32.015542 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:32.015497    5820 generic.go:356] "Generic (PLEG): container finished" podID="98046a89-b091-4e08-9696-985da1473d2a" containerID="aaf14a7e190c6d0ed2f58b6cc337db08cd7eea9fcb9dcaf1cdaa52e2e6ad4a20" exitCode=0
Oct 30 19:31:32.015718 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:32.015568    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf" event={"ID":"98046a89-b091-4e08-9696-985da1473d2a","Type":"ContainerDied","Data":"aaf14a7e190c6d0ed2f58b6cc337db08cd7eea9fcb9dcaf1cdaa52e2e6ad4a20"}
Oct 30 19:31:32.093662 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:32.093611    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf"
Oct 30 19:31:32.128762 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:32.128739    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/98046a89-b091-4e08-9696-985da1473d2a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98046a89-b091-4e08-9696-985da1473d2a-plugins-config-volume\") pod \"98046a89-b091-4e08-9696-985da1473d2a\" (UID: \"98046a89-b091-4e08-9696-985da1473d2a\") "
Oct 30 19:31:32.128982 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:32.128940    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98046a89-b091-4e08-9696-985da1473d2a-kube-api-access-qgl5g\" (UniqueName: \"kubernetes.io/projected/98046a89-b091-4e08-9696-985da1473d2a-kube-api-access-qgl5g\") pod \"98046a89-b091-4e08-9696-985da1473d2a\" (UID: \"98046a89-b091-4e08-9696-985da1473d2a\") "
Oct 30 19:31:32.129155 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:32.129131    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98046a89-b091-4e08-9696-985da1473d2a-plugins-config-volume" pod "98046a89-b091-4e08-9696-985da1473d2a" (UID: "98046a89-b091-4e08-9696-985da1473d2a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:31:32.129396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:32.129378    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98046a89-b091-4e08-9696-985da1473d2a-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:31:32.131720 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:32.131692    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98046a89-b091-4e08-9696-985da1473d2a-kube-api-access-qgl5g" pod "98046a89-b091-4e08-9696-985da1473d2a" (UID: "98046a89-b091-4e08-9696-985da1473d2a"). InnerVolumeSpecName "kube-api-access-qgl5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:31:32.243082 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:32.243044    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgl5g\" (UniqueName: \"kubernetes.io/projected/98046a89-b091-4e08-9696-985da1473d2a-kube-api-access-qgl5g\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:31:32.248741 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:32.246842    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:33.023821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:33.023792    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf"
Oct 30 19:31:33.024318 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:33.023818    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf" event={"ID":"98046a89-b091-4e08-9696-985da1473d2a","Type":"ContainerDied","Data":"facda04052cbd8706b73254528bffe2cd71fac5894079aa6b95cce3f3fdf0463"}
Oct 30 19:31:33.024318 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:33.023886    5820 scope.go:122] "RemoveContainer" containerID="aaf14a7e190c6d0ed2f58b6cc337db08cd7eea9fcb9dcaf1cdaa52e2e6ad4a20"
Oct 30 19:31:33.193602 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:33.193562    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf"]
Oct 30 19:31:33.242836 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:33.242802    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-200c46-864c98dd95-bjwmf"]
Oct 30 19:31:33.281724 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:33.281643    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:33.437621 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:33.437588    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="98046a89-b091-4e08-9696-985da1473d2a" path="/var/lib/kubelet/pods/98046a89-b091-4e08-9696-985da1473d2a/volumes"
Oct 30 19:31:34.179140 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:34.179109    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"]
Oct 30 19:31:34.341504 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:34.341470    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:37.038019 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:37.037859    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:37.213394 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:37.213342    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:38.208178 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:31:38.208144    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:31:38.678619 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:38.678585    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:39.173623 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.173589    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"]
Oct 30 19:31:39.174211 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.174197    5820 cpu_manager.go:414] "containerMap: removing container" podUID="98046a89-b091-4e08-9696-985da1473d2a" containerName="payload-pre-processing"
Oct 30 19:31:39.174279 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.174212    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="98046a89-b091-4e08-9696-985da1473d2a" containerName="payload-pre-processing"
Oct 30 19:31:39.174325 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.174315    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="98046a89-b091-4e08-9696-985da1473d2a" containerName="payload-pre-processing"
Oct 30 19:31:39.177422 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.177396    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.235281 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.235238    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-cdd844-openshift-default-dockercfg-72qzd\""
Oct 30 19:31:39.291903 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.291864    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"]
Oct 30 19:31:39.321639 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.321601    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:39.346022 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.345942    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-credential-socket\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.346209 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.346084    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-workload-socket\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.346209 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.346160    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-workload-certs\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.346336 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.346225    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-istio-data\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.346336 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.346254    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/664f1334-638a-4581-9c84-5a2ea290ec81-istiod-ca-cert\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.346336 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.346270    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85bsq\" (UniqueName: \"kubernetes.io/projected/664f1334-638a-4581-9c84-5a2ea290ec81-kube-api-access-85bsq\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.346499 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.346361    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-istio-envoy\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.346499 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.346460    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/664f1334-638a-4581-9c84-5a2ea290ec81-istio-podinfo\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.346499 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.346495    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/664f1334-638a-4581-9c84-5a2ea290ec81-istio-token\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.448324 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.448252    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-workload-socket\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.448490 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.448357    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-workload-certs\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.448490 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.448410    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-istio-data\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.448490 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.448451    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/664f1334-638a-4581-9c84-5a2ea290ec81-istiod-ca-cert\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.448490 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.448477    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85bsq\" (UniqueName: \"kubernetes.io/projected/664f1334-638a-4581-9c84-5a2ea290ec81-kube-api-access-85bsq\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.448738 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.448537    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-istio-envoy\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.448738 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.448595    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/664f1334-638a-4581-9c84-5a2ea290ec81-istio-podinfo\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.448738 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.448640    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/664f1334-638a-4581-9c84-5a2ea290ec81-istio-token\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.448738 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.448681    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-credential-socket\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.448738 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.448692    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-workload-socket\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.448738 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.448718    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-workload-certs\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.449084 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.448801    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-istio-data\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.449084 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.449059    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-credential-socket\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.449281 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.449255    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/664f1334-638a-4581-9c84-5a2ea290ec81-istiod-ca-cert\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.451000 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.450977    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-istio-envoy\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.451359 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.451334    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/664f1334-638a-4581-9c84-5a2ea290ec81-istio-podinfo\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.502347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.502319    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:39.510222 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.510193    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/664f1334-638a-4581-9c84-5a2ea290ec81-istio-token\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.510367 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.510286    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85bsq\" (UniqueName: \"kubernetes.io/projected/664f1334-638a-4581-9c84-5a2ea290ec81-kube-api-access-85bsq\") pod \"e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") " pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:39.842156 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.842123    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-cdd844-openshift-default-dockercfg-72qzd\""
Oct 30 19:31:39.851470 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:39.851446    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:40.284242 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:40.284215    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"]
Oct 30 19:31:40.285525 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:31:40.285488    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod664f1334_638a_4581_9c84_5a2ea290ec81.slice/crio-b2c360a42a74b7711378191d1465589fddd32325a51663c940ff46af7cccb3c7 WatchSource:0}: Error finding container b2c360a42a74b7711378191d1465589fddd32325a51663c940ff46af7cccb3c7: Status 404 returned error can't find the container with id b2c360a42a74b7711378191d1465589fddd32325a51663c940ff46af7cccb3c7
Oct 30 19:31:40.288126 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:40.288092    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:31:40.288231 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:40.288158    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:31:40.288231 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:40.288188    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:31:41.081377 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:41.081342    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x" event={"ID":"664f1334-638a-4581-9c84-5a2ea290ec81","Type":"ContainerStarted","Data":"2eb0cdd4b9da0e2f19c6ac3d6ea85af27aab70677f49711a538c0ad9b5f9c2b7"}
Oct 30 19:31:41.081570 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:41.081407    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x" event={"ID":"664f1334-638a-4581-9c84-5a2ea290ec81","Type":"ContainerStarted","Data":"b2c360a42a74b7711378191d1465589fddd32325a51663c940ff46af7cccb3c7"}
Oct 30 19:31:41.211431 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:41.211303    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x" podStartSLOduration=2.211284416 podStartE2EDuration="2.211284416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:41.208629281 +0000 UTC m=+3421.920377978" watchObservedRunningTime="2026-10-30 19:31:41.211284416 +0000 UTC m=+3421.923033113"
Oct 30 19:31:41.852045 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:41.852008    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:41.857978 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:41.857928    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:41.872002 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:41.871974    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:42.051073 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:42.051038    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:42.087447 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:42.087417    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:42.088510 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:42.088490    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:31:44.020281 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:44.020148    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:44.204679 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:44.204577    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:44.382570 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:44.382541    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:47.216276 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:47.216229    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:47.466090 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:47.466005    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:47.665517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:47.665484    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:49.200573 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:49.200537    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk" podUID="90c808d6-a5be-4e0e-979a-69a5cc4571fc" containerName="main" containerID="cri-o://fc6d23923595885e81e058ee73df2cb01a2570cdc54d333e754a320f893777d8" gracePeriod=45
Oct 30 19:31:49.246051 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:49.246020    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk" podUID="90c808d6-a5be-4e0e-979a-69a5cc4571fc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.183:8000/ready\": dial tcp 10.134.0.183:8000: connect: connection refused"
Oct 30 19:31:49.471235 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:49.471212    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:31:49.578339 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:49.578307    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-tmp-dir\") pod \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") "
Oct 30 19:31:49.578525 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:49.578384    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-dshm\" (UniqueName: \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-dshm\") pod \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") "
Oct 30 19:31:49.578525 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:49.578417    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-home\" (UniqueName: \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-home\") pod \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") "
Oct 30 19:31:49.578525 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:49.578448    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-model-cache\") pod \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") "
Oct 30 19:31:49.578525 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:49.578494    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/90c808d6-a5be-4e0e-979a-69a5cc4571fc-tls-certs\" (UniqueName: \"kubernetes.io/secret/90c808d6-a5be-4e0e-979a-69a5cc4571fc-tls-certs\") pod \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") "
Oct 30 19:31:49.578750 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:49.578605    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-tmp-dir" pod "90c808d6-a5be-4e0e-979a-69a5cc4571fc" (UID: "90c808d6-a5be-4e0e-979a-69a5cc4571fc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:31:49.578750 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:49.578636    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90c808d6-a5be-4e0e-979a-69a5cc4571fc-kube-api-access-nr9sj\" (UniqueName: \"kubernetes.io/projected/90c808d6-a5be-4e0e-979a-69a5cc4571fc-kube-api-access-nr9sj\") pod \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\" (UID: \"90c808d6-a5be-4e0e-979a-69a5cc4571fc\") "
Oct 30 19:31:49.578750 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:49.578700    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-home" pod "90c808d6-a5be-4e0e-979a-69a5cc4571fc" (UID: "90c808d6-a5be-4e0e-979a-69a5cc4571fc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:31:49.578750 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:49.578739    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-model-cache" pod "90c808d6-a5be-4e0e-979a-69a5cc4571fc" (UID: "90c808d6-a5be-4e0e-979a-69a5cc4571fc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:31:49.579080 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:49.579059    5820 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-tmp-dir\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:31:49.579080 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:49.579080    5820 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-home\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:31:49.579404 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:49.579089    5820 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-model-cache\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:31:49.580885 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:49.580861    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90c808d6-a5be-4e0e-979a-69a5cc4571fc-tls-certs" pod "90c808d6-a5be-4e0e-979a-69a5cc4571fc" (UID: "90c808d6-a5be-4e0e-979a-69a5cc4571fc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:31:49.581017 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:49.580920    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-dshm" pod "90c808d6-a5be-4e0e-979a-69a5cc4571fc" (UID: "90c808d6-a5be-4e0e-979a-69a5cc4571fc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:31:49.581137 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:49.581118    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90c808d6-a5be-4e0e-979a-69a5cc4571fc-kube-api-access-nr9sj" pod "90c808d6-a5be-4e0e-979a-69a5cc4571fc" (UID: "90c808d6-a5be-4e0e-979a-69a5cc4571fc"). InnerVolumeSpecName "kube-api-access-nr9sj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:31:49.679968 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:49.679913    5820 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90c808d6-a5be-4e0e-979a-69a5cc4571fc-dshm\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:31:49.679968 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:49.679965    5820 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90c808d6-a5be-4e0e-979a-69a5cc4571fc-tls-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:31:49.680252 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:49.679981    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nr9sj\" (UniqueName: \"kubernetes.io/projected/90c808d6-a5be-4e0e-979a-69a5cc4571fc-kube-api-access-nr9sj\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:31:50.166329 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:50.166296    5820 generic.go:356] "Generic (PLEG): container finished" podID="90c808d6-a5be-4e0e-979a-69a5cc4571fc" containerID="fc6d23923595885e81e058ee73df2cb01a2570cdc54d333e754a320f893777d8" exitCode=0
Oct 30 19:31:50.166517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:50.166348    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk" event={"ID":"90c808d6-a5be-4e0e-979a-69a5cc4571fc","Type":"ContainerDied","Data":"fc6d23923595885e81e058ee73df2cb01a2570cdc54d333e754a320f893777d8"}
Oct 30 19:31:50.166517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:50.166382    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk" event={"ID":"90c808d6-a5be-4e0e-979a-69a5cc4571fc","Type":"ContainerDied","Data":"d23ca2ed5d5b9453356a1006526793da258dc7736c25e2c07befe6e7f1795e15"}
Oct 30 19:31:50.166517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:50.166383    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"
Oct 30 19:31:50.166517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:50.166397    5820 scope.go:122] "RemoveContainer" containerID="fc6d23923595885e81e058ee73df2cb01a2570cdc54d333e754a320f893777d8"
Oct 30 19:31:50.177236 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:50.177216    5820 scope.go:122] "RemoveContainer" containerID="fc6d23923595885e81e058ee73df2cb01a2570cdc54d333e754a320f893777d8"
Oct 30 19:31:50.177468 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:31:50.177446    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc6d23923595885e81e058ee73df2cb01a2570cdc54d333e754a320f893777d8\": container with ID starting with fc6d23923595885e81e058ee73df2cb01a2570cdc54d333e754a320f893777d8 not found: ID does not exist" containerID="fc6d23923595885e81e058ee73df2cb01a2570cdc54d333e754a320f893777d8"
Oct 30 19:31:50.177546 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:50.177476    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc6d23923595885e81e058ee73df2cb01a2570cdc54d333e754a320f893777d8"} err="failed to get container status \"fc6d23923595885e81e058ee73df2cb01a2570cdc54d333e754a320f893777d8\": rpc error: code = NotFound desc = could not find container \"fc6d23923595885e81e058ee73df2cb01a2570cdc54d333e754a320f893777d8\": container with ID starting with fc6d23923595885e81e058ee73df2cb01a2570cdc54d333e754a320f893777d8 not found: ID does not exist"
Oct 30 19:31:50.330123 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:50.330095    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"]
Oct 30 19:31:50.379318 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:50.379292    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w1-23302c/auth-test-model-23302c-kserve-58ff984649-8gpvk"]
Oct 30 19:31:51.436802 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:51.436772    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="90c808d6-a5be-4e0e-979a-69a5cc4571fc" path="/var/lib/kubelet/pods/90c808d6-a5be-4e0e-979a-69a5cc4571fc/volumes"
Oct 30 19:31:52.127915 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:52.127881    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"]
Oct 30 19:31:52.128543 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:52.128527    5820 cpu_manager.go:414] "containerMap: removing container" podUID="90c808d6-a5be-4e0e-979a-69a5cc4571fc" containerName="main"
Oct 30 19:31:52.128595 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:52.128546    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90c808d6-a5be-4e0e-979a-69a5cc4571fc" containerName="main"
Oct 30 19:31:52.128686 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:52.128676    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="90c808d6-a5be-4e0e-979a-69a5cc4571fc" containerName="main"
Oct 30 19:31:52.133403 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:52.133383    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"
Oct 30 19:31:52.191519 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:52.191484    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"]
Oct 30 19:31:52.212673 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:52.212648    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af249134-3666-43cf-b772-8178f036a60d-maas-api-tls\") pod \"maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9\" (UID: \"af249134-3666-43cf-b772-8178f036a60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"
Oct 30 19:31:52.212807 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:52.212738    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af249134-3666-43cf-b772-8178f036a60d-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9\" (UID: \"af249134-3666-43cf-b772-8178f036a60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"
Oct 30 19:31:52.212807 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:52.212759    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af249134-3666-43cf-b772-8178f036a60d-metrics-certs\") pod \"maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9\" (UID: \"af249134-3666-43cf-b772-8178f036a60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"
Oct 30 19:31:52.212989 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:52.212961    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx2cp\" (UniqueName: \"kubernetes.io/projected/af249134-3666-43cf-b772-8178f036a60d-kube-api-access-hx2cp\") pod \"maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9\" (UID: \"af249134-3666-43cf-b772-8178f036a60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"
Oct 30 19:31:52.237820 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:52.237790    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-7fdf6b-serving-cert\""
Oct 30 19:31:52.314309 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:52.314281    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af249134-3666-43cf-b772-8178f036a60d-maas-api-tls\") pod \"maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9\" (UID: \"af249134-3666-43cf-b772-8178f036a60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"
Oct 30 19:31:52.314473 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:52.314370    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af249134-3666-43cf-b772-8178f036a60d-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9\" (UID: \"af249134-3666-43cf-b772-8178f036a60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"
Oct 30 19:31:52.314473 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:52.314396    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af249134-3666-43cf-b772-8178f036a60d-metrics-certs\") pod \"maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9\" (UID: \"af249134-3666-43cf-b772-8178f036a60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"
Oct 30 19:31:52.314473 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:31:52.314430    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-serving-cert: secret "maas-api-e2e-extmod-a-7fdf6b-serving-cert" not found
Oct 30 19:31:52.314631 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:31:52.314503    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af249134-3666-43cf-b772-8178f036a60d-maas-api-tls podName:af249134-3666-43cf-b772-8178f036a60d nodeName:}" failed. No retries permitted until 2026-10-30 19:31:52.81447009 +0000 UTC m=+3433.526218768 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/af249134-3666-43cf-b772-8178f036a60d-maas-api-tls") pod "maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9" (UID: "af249134-3666-43cf-b772-8178f036a60d") : secret "maas-api-e2e-extmod-a-7fdf6b-serving-cert" not found
Oct 30 19:31:52.314631 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:52.314577    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx2cp\" (UniqueName: \"kubernetes.io/projected/af249134-3666-43cf-b772-8178f036a60d-kube-api-access-hx2cp\") pod \"maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9\" (UID: \"af249134-3666-43cf-b772-8178f036a60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"
Oct 30 19:31:52.315142 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:52.315121    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af249134-3666-43cf-b772-8178f036a60d-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9\" (UID: \"af249134-3666-43cf-b772-8178f036a60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"
Oct 30 19:31:52.317037 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:52.317020    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af249134-3666-43cf-b772-8178f036a60d-metrics-certs\") pod \"maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9\" (UID: \"af249134-3666-43cf-b772-8178f036a60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"
Oct 30 19:31:52.383110 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:52.382993    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx2cp\" (UniqueName: \"kubernetes.io/projected/af249134-3666-43cf-b772-8178f036a60d-kube-api-access-hx2cp\") pod \"maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9\" (UID: \"af249134-3666-43cf-b772-8178f036a60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"
Oct 30 19:31:52.406923 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:52.406891    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:52.678448 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:52.678363    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:52.820882 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:52.820847    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af249134-3666-43cf-b772-8178f036a60d-maas-api-tls\") pod \"maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9\" (UID: \"af249134-3666-43cf-b772-8178f036a60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"
Oct 30 19:31:52.823486 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:52.823463    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af249134-3666-43cf-b772-8178f036a60d-maas-api-tls\") pod \"maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9\" (UID: \"af249134-3666-43cf-b772-8178f036a60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"
Oct 30 19:31:53.047393 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:53.047316    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"
Oct 30 19:31:53.425048 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:53.425024    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"]
Oct 30 19:31:53.426657 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:31:53.426630    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf249134_3666_43cf_b772_8178f036a60d.slice/crio-0022e4ea90c76c9985855205c2e66dab841ec74d31c1380ab86cc3bffa6d9557 WatchSource:0}: Error finding container 0022e4ea90c76c9985855205c2e66dab841ec74d31c1380ab86cc3bffa6d9557: Status 404 returned error can't find the container with id 0022e4ea90c76c9985855205c2e66dab841ec74d31c1380ab86cc3bffa6d9557
Oct 30 19:31:54.191635 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:54.191592    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9" event={"ID":"af249134-3666-43cf-b772-8178f036a60d","Type":"ContainerStarted","Data":"0022e4ea90c76c9985855205c2e66dab841ec74d31c1380ab86cc3bffa6d9557"}
Oct 30 19:31:55.198517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:55.198485    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9" event={"ID":"af249134-3666-43cf-b772-8178f036a60d","Type":"ContainerStarted","Data":"7fb52710100ff11d538ea514e078a5da6a96959c09275e384c661f6e64bdc1ca"}
Oct 30 19:31:55.198877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:55.198600    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"
Oct 30 19:31:55.316472 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:55.316409    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9" podStartSLOduration=2.25431192 podStartE2EDuration="3.316391522s" totalImagesPullingTime="1.062079602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:55.315530475 +0000 UTC m=+3436.027279186" watchObservedRunningTime="2026-10-30 19:31:55.316391522 +0000 UTC m=+3436.028140222"
Oct 30 19:31:57.545354 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:31:57.545313    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:01.210801 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:01.210774    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"
Oct 30 19:32:02.232351 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:02.232240    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:05.805555 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:05.805513    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:06.078502 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:06.078407    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:07.051165 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:07.051128    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:08.203848 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:32:08.203812    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:32:11.852565 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:11.852525    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:16.640186 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:16.640147    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:20.714498 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:20.714464    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"]
Oct 30 19:32:20.715294 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:20.715255    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8" podUID="6038d170-874b-48fc-8665-982bff9cdc30" containerName="maas-api" containerID="cri-o://4068a283c19eea38f4a2a83ee9c3266d5dee8fc220c4d8260d9e60bc1cc09910" gracePeriod=30
Oct 30 19:32:20.855734 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:20.855696    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:21.019421 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:21.019394    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"
Oct 30 19:32:21.055622 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:21.055596    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6038d170-874b-48fc-8665-982bff9cdc30-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6038d170-874b-48fc-8665-982bff9cdc30-maas-api-tls\") pod \"6038d170-874b-48fc-8665-982bff9cdc30\" (UID: \"6038d170-874b-48fc-8665-982bff9cdc30\") "
Oct 30 19:32:21.055799 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:21.055692    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6038d170-874b-48fc-8665-982bff9cdc30-kube-api-access-48bkm\" (UniqueName: \"kubernetes.io/projected/6038d170-874b-48fc-8665-982bff9cdc30-kube-api-access-48bkm\") pod \"6038d170-874b-48fc-8665-982bff9cdc30\" (UID: \"6038d170-874b-48fc-8665-982bff9cdc30\") "
Oct 30 19:32:21.055869 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:21.055818    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6038d170-874b-48fc-8665-982bff9cdc30-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6038d170-874b-48fc-8665-982bff9cdc30-openshift-service-ca\") pod \"6038d170-874b-48fc-8665-982bff9cdc30\" (UID: \"6038d170-874b-48fc-8665-982bff9cdc30\") "
Oct 30 19:32:21.055931 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:21.055901    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6038d170-874b-48fc-8665-982bff9cdc30-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6038d170-874b-48fc-8665-982bff9cdc30-metrics-certs\") pod \"6038d170-874b-48fc-8665-982bff9cdc30\" (UID: \"6038d170-874b-48fc-8665-982bff9cdc30\") "
Oct 30 19:32:21.056235 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:21.056205    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6038d170-874b-48fc-8665-982bff9cdc30-openshift-service-ca" pod "6038d170-874b-48fc-8665-982bff9cdc30" (UID: "6038d170-874b-48fc-8665-982bff9cdc30"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:21.056649 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:21.056604    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6038d170-874b-48fc-8665-982bff9cdc30-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.057842 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:21.057814    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6038d170-874b-48fc-8665-982bff9cdc30-maas-api-tls" pod "6038d170-874b-48fc-8665-982bff9cdc30" (UID: "6038d170-874b-48fc-8665-982bff9cdc30"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:21.057997 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:21.057979    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6038d170-874b-48fc-8665-982bff9cdc30-kube-api-access-48bkm" pod "6038d170-874b-48fc-8665-982bff9cdc30" (UID: "6038d170-874b-48fc-8665-982bff9cdc30"). InnerVolumeSpecName "kube-api-access-48bkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:21.058063 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:21.058051    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6038d170-874b-48fc-8665-982bff9cdc30-metrics-certs" pod "6038d170-874b-48fc-8665-982bff9cdc30" (UID: "6038d170-874b-48fc-8665-982bff9cdc30"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:21.157653 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:21.157624    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6038d170-874b-48fc-8665-982bff9cdc30-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.157653 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:21.157651    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48bkm\" (UniqueName: \"kubernetes.io/projected/6038d170-874b-48fc-8665-982bff9cdc30-kube-api-access-48bkm\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.157846 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:21.157662    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6038d170-874b-48fc-8665-982bff9cdc30-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.361670 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:21.361636    5820 generic.go:356] "Generic (PLEG): container finished" podID="6038d170-874b-48fc-8665-982bff9cdc30" containerID="4068a283c19eea38f4a2a83ee9c3266d5dee8fc220c4d8260d9e60bc1cc09910" exitCode=0
Oct 30 19:32:21.361884 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:21.361682    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"
Oct 30 19:32:21.361884 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:21.361712    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8" event={"ID":"6038d170-874b-48fc-8665-982bff9cdc30","Type":"ContainerDied","Data":"4068a283c19eea38f4a2a83ee9c3266d5dee8fc220c4d8260d9e60bc1cc09910"}
Oct 30 19:32:21.361884 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:21.361803    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8" event={"ID":"6038d170-874b-48fc-8665-982bff9cdc30","Type":"ContainerDied","Data":"14b4e7350da86ccff4d6c067e9e27f7a3b0ff8e4c097ff8a0a0e5d222364bf0b"}
Oct 30 19:32:21.361884 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:21.361833    5820 scope.go:122] "RemoveContainer" containerID="4068a283c19eea38f4a2a83ee9c3266d5dee8fc220c4d8260d9e60bc1cc09910"
Oct 30 19:32:21.372841 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:21.372824    5820 scope.go:122] "RemoveContainer" containerID="4068a283c19eea38f4a2a83ee9c3266d5dee8fc220c4d8260d9e60bc1cc09910"
Oct 30 19:32:21.373137 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:32:21.373117    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4068a283c19eea38f4a2a83ee9c3266d5dee8fc220c4d8260d9e60bc1cc09910\": container with ID starting with 4068a283c19eea38f4a2a83ee9c3266d5dee8fc220c4d8260d9e60bc1cc09910 not found: ID does not exist" containerID="4068a283c19eea38f4a2a83ee9c3266d5dee8fc220c4d8260d9e60bc1cc09910"
Oct 30 19:32:21.373213 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:21.373148    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4068a283c19eea38f4a2a83ee9c3266d5dee8fc220c4d8260d9e60bc1cc09910"} err="failed to get container status \"4068a283c19eea38f4a2a83ee9c3266d5dee8fc220c4d8260d9e60bc1cc09910\": rpc error: code = NotFound desc = could not find container \"4068a283c19eea38f4a2a83ee9c3266d5dee8fc220c4d8260d9e60bc1cc09910\": container with ID starting with 4068a283c19eea38f4a2a83ee9c3266d5dee8fc220c4d8260d9e60bc1cc09910 not found: ID does not exist"
Oct 30 19:32:21.541899 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:21.541864    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"]
Oct 30 19:32:21.564707 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:21.564657    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-200c46-6cd96d97dd-m4vm8"]
Oct 30 19:32:22.420248 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:22.420216    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:22.676742 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:22.676646    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:22.862339 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:22.862289    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:23.390630 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:23.390593    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:23.436849 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:23.436819    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6038d170-874b-48fc-8665-982bff9cdc30" path="/var/lib/kubelet/pods/6038d170-874b-48fc-8665-982bff9cdc30/volumes"
Oct 30 19:32:28.439682 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:28.435872    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:29.910455 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:29.910417    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:30.092028 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:30.091936    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:30.291372 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:30.291270    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:31.098217 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:31.098175    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:31.626190 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:31.626153    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:31.938025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:31.937906    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:33.449297 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:33.449265    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:34.177613 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:34.177579    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:34.365986 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:34.365920    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:34.546416 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:34.546332    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:34.705187 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:34.705157    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"]
Oct 30 19:32:34.802847 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:34.802793    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:35.913876 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:35.913835    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:36.530568 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:36.530524    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:36.810410 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:36.810379    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:38.203642 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:32:38.203607    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:32:39.079579 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:39.079543    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:41.545386 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:41.545350    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:41.772475 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:41.772431    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:44.387001 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:44.386088    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:44.774745 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:44.774658    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:44.961561 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:44.961523    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:49.536401 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:49.536356    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:49.727190 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:49.727152    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6" podUID="bfdd1d8f-7962-4456-9314-666d19e646d6" containerName="main" containerID="cri-o://827d33f54cfd631603daf7881e75721edb6d0397809945935f3e1a689745d7b8" gracePeriod=45
Oct 30 19:32:50.010268 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.010233    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:32:50.115173 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.115085    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-model-cache\") pod \"bfdd1d8f-7962-4456-9314-666d19e646d6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") "
Oct 30 19:32:50.115173 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.115173    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-dshm\") pod \"bfdd1d8f-7962-4456-9314-666d19e646d6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") "
Oct 30 19:32:50.115427 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.115204    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-home\" (UniqueName: \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-home\") pod \"bfdd1d8f-7962-4456-9314-666d19e646d6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") "
Oct 30 19:32:50.115427 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.115278    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-tmp-dir\") pod \"bfdd1d8f-7962-4456-9314-666d19e646d6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") "
Oct 30 19:32:50.115427 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.115307    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bfdd1d8f-7962-4456-9314-666d19e646d6-kube-api-access-9qt6m\" (UniqueName: \"kubernetes.io/projected/bfdd1d8f-7962-4456-9314-666d19e646d6-kube-api-access-9qt6m\") pod \"bfdd1d8f-7962-4456-9314-666d19e646d6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") "
Oct 30 19:32:50.115427 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.115338    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-model-cache" pod "bfdd1d8f-7962-4456-9314-666d19e646d6" (UID: "bfdd1d8f-7962-4456-9314-666d19e646d6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:50.115427 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.115403    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bfdd1d8f-7962-4456-9314-666d19e646d6-tls-certs\" (UniqueName: \"kubernetes.io/secret/bfdd1d8f-7962-4456-9314-666d19e646d6-tls-certs\") pod \"bfdd1d8f-7962-4456-9314-666d19e646d6\" (UID: \"bfdd1d8f-7962-4456-9314-666d19e646d6\") "
Oct 30 19:32:50.115742 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.115501    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-tmp-dir" pod "bfdd1d8f-7962-4456-9314-666d19e646d6" (UID: "bfdd1d8f-7962-4456-9314-666d19e646d6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:50.115742 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.115544    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-home" pod "bfdd1d8f-7962-4456-9314-666d19e646d6" (UID: "bfdd1d8f-7962-4456-9314-666d19e646d6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:50.116011 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.115988    5820 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-model-cache\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:50.116093 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.116016    5820 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-home\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:50.116093 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.116030    5820 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-tmp-dir\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:50.117692 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.117670    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfdd1d8f-7962-4456-9314-666d19e646d6-tls-certs" pod "bfdd1d8f-7962-4456-9314-666d19e646d6" (UID: "bfdd1d8f-7962-4456-9314-666d19e646d6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:50.117793 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.117704    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-dshm" pod "bfdd1d8f-7962-4456-9314-666d19e646d6" (UID: "bfdd1d8f-7962-4456-9314-666d19e646d6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:50.117936 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.117918    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfdd1d8f-7962-4456-9314-666d19e646d6-kube-api-access-9qt6m" pod "bfdd1d8f-7962-4456-9314-666d19e646d6" (UID: "bfdd1d8f-7962-4456-9314-666d19e646d6"). InnerVolumeSpecName "kube-api-access-9qt6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:50.216748 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.216713    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qt6m\" (UniqueName: \"kubernetes.io/projected/bfdd1d8f-7962-4456-9314-666d19e646d6-kube-api-access-9qt6m\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:50.216748 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.216744    5820 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfdd1d8f-7962-4456-9314-666d19e646d6-tls-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:50.216941 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.216759    5820 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfdd1d8f-7962-4456-9314-666d19e646d6-dshm\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:50.569195 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.569162    5820 generic.go:356] "Generic (PLEG): container finished" podID="bfdd1d8f-7962-4456-9314-666d19e646d6" containerID="827d33f54cfd631603daf7881e75721edb6d0397809945935f3e1a689745d7b8" exitCode=0
Oct 30 19:32:50.569644 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.569204    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"
Oct 30 19:32:50.569644 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.569244    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6" event={"ID":"bfdd1d8f-7962-4456-9314-666d19e646d6","Type":"ContainerDied","Data":"827d33f54cfd631603daf7881e75721edb6d0397809945935f3e1a689745d7b8"}
Oct 30 19:32:50.569644 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.569285    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6" event={"ID":"bfdd1d8f-7962-4456-9314-666d19e646d6","Type":"ContainerDied","Data":"9b71d32f0e794aeb785a5e620deab53781b3babc4c8b3d49e33e24fe389b3e54"}
Oct 30 19:32:50.569644 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.569301    5820 scope.go:122] "RemoveContainer" containerID="827d33f54cfd631603daf7881e75721edb6d0397809945935f3e1a689745d7b8"
Oct 30 19:32:50.579678 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.579659    5820 scope.go:122] "RemoveContainer" containerID="827d33f54cfd631603daf7881e75721edb6d0397809945935f3e1a689745d7b8"
Oct 30 19:32:50.579937 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:32:50.579919    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"827d33f54cfd631603daf7881e75721edb6d0397809945935f3e1a689745d7b8\": container with ID starting with 827d33f54cfd631603daf7881e75721edb6d0397809945935f3e1a689745d7b8 not found: ID does not exist" containerID="827d33f54cfd631603daf7881e75721edb6d0397809945935f3e1a689745d7b8"
Oct 30 19:32:50.580096 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.580075    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"827d33f54cfd631603daf7881e75721edb6d0397809945935f3e1a689745d7b8"} err="failed to get container status \"827d33f54cfd631603daf7881e75721edb6d0397809945935f3e1a689745d7b8\": rpc error: code = NotFound desc = could not find container \"827d33f54cfd631603daf7881e75721edb6d0397809945935f3e1a689745d7b8\": container with ID starting with 827d33f54cfd631603daf7881e75721edb6d0397809945935f3e1a689745d7b8 not found: ID does not exist"
Oct 30 19:32:50.733998 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.733968    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"]
Oct 30 19:32:50.784010 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:50.783982    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w0-a56c7c/auth-test-model-a56c7c-kserve-7d7f97cc57-4mjl6"]
Oct 30 19:32:51.436583 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:51.436549    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bfdd1d8f-7962-4456-9314-666d19e646d6" path="/var/lib/kubelet/pods/bfdd1d8f-7962-4456-9314-666d19e646d6/volumes"
Oct 30 19:32:54.315623 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:54.315527    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:54.770504 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:54.770422    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:58.843476 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:58.843441    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"]
Oct 30 19:32:58.844067 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:58.844053    5820 cpu_manager.go:414] "containerMap: removing container" podUID="bfdd1d8f-7962-4456-9314-666d19e646d6" containerName="main"
Oct 30 19:32:58.844124 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:58.844068    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bfdd1d8f-7962-4456-9314-666d19e646d6" containerName="main"
Oct 30 19:32:58.844124 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:58.844100    5820 cpu_manager.go:414] "containerMap: removing container" podUID="6038d170-874b-48fc-8665-982bff9cdc30" containerName="maas-api"
Oct 30 19:32:58.844124 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:58.844105    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6038d170-874b-48fc-8665-982bff9cdc30" containerName="maas-api"
Oct 30 19:32:58.844231 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:58.844212    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="bfdd1d8f-7962-4456-9314-666d19e646d6" containerName="main"
Oct 30 19:32:58.844231 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:58.844225    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="6038d170-874b-48fc-8665-982bff9cdc30" containerName="maas-api"
Oct 30 19:32:58.848855 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:58.848837    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"
Oct 30 19:32:58.902936 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:58.902904    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-cdd844-serving-cert\""
Oct 30 19:32:58.910810 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:58.910784    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-maas-api-tls\") pod \"maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26\" (UID: \"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"
Oct 30 19:32:58.910934 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:58.910847    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-metrics-certs\") pod \"maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26\" (UID: \"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"
Oct 30 19:32:58.911013 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:58.910979    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkldj\" (UniqueName: \"kubernetes.io/projected/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-kube-api-access-pkldj\") pod \"maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26\" (UID: \"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"
Oct 30 19:32:58.911113 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:58.911096    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26\" (UID: \"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"
Oct 30 19:32:58.955209 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:58.955182    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"]
Oct 30 19:32:59.011803 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:59.011775    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-metrics-certs\") pod \"maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26\" (UID: \"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"
Oct 30 19:32:59.011969 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:59.011822    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkldj\" (UniqueName: \"kubernetes.io/projected/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-kube-api-access-pkldj\") pod \"maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26\" (UID: \"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"
Oct 30 19:32:59.011969 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:59.011903    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26\" (UID: \"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"
Oct 30 19:32:59.012110 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:59.012049    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-maas-api-tls\") pod \"maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26\" (UID: \"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"
Oct 30 19:32:59.012174 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:32:59.012152    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-serving-cert: secret "maas-api-e2e-extmod-a-cdd844-serving-cert" not found
Oct 30 19:32:59.012265 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:32:59.012243    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-maas-api-tls podName:1ad9d9e9-639f-49a3-a53b-e0dda71e87b4 nodeName:}" failed. No retries permitted until 2026-10-30 19:32:59.51222211 +0000 UTC m=+3500.223970804 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-maas-api-tls") pod "maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26" (UID: "1ad9d9e9-639f-49a3-a53b-e0dda71e87b4") : secret "maas-api-e2e-extmod-a-cdd844-serving-cert" not found
Oct 30 19:32:59.012619 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:59.012601    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26\" (UID: \"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"
Oct 30 19:32:59.014486 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:59.014463    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-metrics-certs\") pod \"maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26\" (UID: \"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"
Oct 30 19:32:59.073758 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:59.073732    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:59.075501 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:59.075476    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkldj\" (UniqueName: \"kubernetes.io/projected/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-kube-api-access-pkldj\") pod \"maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26\" (UID: \"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"
Oct 30 19:32:59.303151 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:59.303121    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:59.484104 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:59.484068    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:59.518123 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:59.518091    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-maas-api-tls\") pod \"maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26\" (UID: \"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"
Oct 30 19:32:59.520702 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:59.520681    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-maas-api-tls\") pod \"maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26\" (UID: \"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"
Oct 30 19:32:59.763150 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:32:59.763076    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"
Oct 30 19:33:00.144763 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:00.144733    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"]
Oct 30 19:33:00.146659 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:33:00.146630    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ad9d9e9_639f_49a3_a53b_e0dda71e87b4.slice/crio-50d9d4efff6a94082af3b07f6ae7988ea7ea126beab5d6d3b3a6dd49c606ef92 WatchSource:0}: Error finding container 50d9d4efff6a94082af3b07f6ae7988ea7ea126beab5d6d3b3a6dd49c606ef92: Status 404 returned error can't find the container with id 50d9d4efff6a94082af3b07f6ae7988ea7ea126beab5d6d3b3a6dd49c606ef92
Oct 30 19:33:00.631398 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:00.631359    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26" event={"ID":"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4","Type":"ContainerStarted","Data":"50d9d4efff6a94082af3b07f6ae7988ea7ea126beab5d6d3b3a6dd49c606ef92"}
Oct 30 19:33:01.640193 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:01.640155    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26" event={"ID":"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4","Type":"ContainerStarted","Data":"5eb73dbb33cd12028ccedbcea8b9fe8c6deb7e3156f9e0eb4ec97c35b103602a"}
Oct 30 19:33:01.640683 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:01.640257    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"
Oct 30 19:33:01.759687 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:01.759602    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26" podStartSLOduration=2.854757659 podStartE2EDuration="3.759583233s" totalImagesPullingTime="904.825574ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:01.757108494 +0000 UTC m=+3502.468857190" watchObservedRunningTime="2026-10-30 19:33:01.759583233 +0000 UTC m=+3502.471331932"
Oct 30 19:33:04.489879 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:04.489846    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:04.978320 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:04.978287    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:05.441482 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:05.441443    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:05.949349 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:05.949319    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"]
Oct 30 19:33:05.953392 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:05.953372    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.007479 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.007379    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w4-0f83f5-openshift-default-dockercfg-p2wns\""
Oct 30 19:33:06.061213 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.061167    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"]
Oct 30 19:33:06.093099 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.093065    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:06.107049 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.107014    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-workload-socket\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.107384 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.107357    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-credential-socket\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.107578 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.107562    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-workload-certs\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.118357 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.107752    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6c5724f-a1ce-448e-9535-3bbc49743b18-istiod-ca-cert\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.118357 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.107792    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-envoy\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.118357 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.107823    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdll2\" (UniqueName: \"kubernetes.io/projected/c6c5724f-a1ce-448e-9535-3bbc49743b18-kube-api-access-mdll2\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.118357 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.107890    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-data\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.118357 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.108098    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-podinfo\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.118357 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.108249    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-token\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.209227 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.209147    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-podinfo\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.209389 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.209265    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-token\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.209389 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.209322    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-workload-socket\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.209389 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.209357    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-credential-socket\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.209389 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.209382    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-workload-certs\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.209605 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.209425    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6c5724f-a1ce-448e-9535-3bbc49743b18-istiod-ca-cert\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.209605 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.209588    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-envoy\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.209717 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.209637    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdll2\" (UniqueName: \"kubernetes.io/projected/c6c5724f-a1ce-448e-9535-3bbc49743b18-kube-api-access-mdll2\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.209778 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.209716    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-data\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.209778 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.209764    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-workload-socket\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.209869 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.209825    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-credential-socket\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.210194 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.210171    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-workload-certs\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.210474 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.210423    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-data\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.210698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.210673    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6c5724f-a1ce-448e-9535-3bbc49743b18-istiod-ca-cert\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.212162 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.212141    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-podinfo\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.212241 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.212142    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-envoy\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.270705 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.270667    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-token\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.270893 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.270799    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdll2\" (UniqueName: \"kubernetes.io/projected/c6c5724f-a1ce-448e-9535-3bbc49743b18-kube-api-access-mdll2\") pod \"e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") " pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:06.325020 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.324982    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:06.503736 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.503651    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:06.567347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:06.567314    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:07.000760 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:07.000732    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"]
Oct 30 19:33:07.002029 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:33:07.002005    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6c5724f_a1ce_448e_9535_3bbc49743b18.slice/crio-1f7b0c1dcb40fa23a02807ec03eac0a77fe8b988087bd4ee604ad727905918df WatchSource:0}: Error finding container 1f7b0c1dcb40fa23a02807ec03eac0a77fe8b988087bd4ee604ad727905918df: Status 404 returned error can't find the container with id 1f7b0c1dcb40fa23a02807ec03eac0a77fe8b988087bd4ee604ad727905918df
Oct 30 19:33:07.004763 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:07.004726    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:33:07.004856 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:07.004796    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:33:07.004856 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:07.004824    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:33:07.651573 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:07.651547    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"
Oct 30 19:33:07.678484 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:07.678453    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x" event={"ID":"c6c5724f-a1ce-448e-9535-3bbc49743b18","Type":"ContainerStarted","Data":"2fe5dcdb9e6357bd512e6ff42239be39f43fe0c0a50110c34a622d418eff7458"}
Oct 30 19:33:07.678663 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:07.678514    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x" event={"ID":"c6c5724f-a1ce-448e-9535-3bbc49743b18","Type":"ContainerStarted","Data":"1f7b0c1dcb40fa23a02807ec03eac0a77fe8b988087bd4ee604ad727905918df"}
Oct 30 19:33:07.937576 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:07.937470    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x" podStartSLOduration=2.937453174 podStartE2EDuration="2.937453174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:07.935816087 +0000 UTC m=+3508.647564785" watchObservedRunningTime="2026-10-30 19:33:07.937453174 +0000 UTC m=+3508.649201875"
Oct 30 19:33:08.203915 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:33:08.203825    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:33:08.567469 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:08.567441    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:08.573223 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:08.573201    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:08.685293 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:08.685261    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:08.686844 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:08.686820    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:33:09.358920 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:09.358879    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:10.806122 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:10.806069    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:11.042175 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:11.042132    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:14.183823 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:14.183781    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:14.376305 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:14.376266    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:14.862064 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:14.848939    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:17.181424 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:17.181381    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:18.511868 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:18.511830    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:19.194188 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:19.194147    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:19.434192 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:19.434155    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:20.078501 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:20.078467    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:23.757563 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:23.757527    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:23.812163 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:23.812132    5820 generic.go:356] "Generic (PLEG): container finished" podID="664f1334-638a-4581-9c84-5a2ea290ec81" containerID="2eb0cdd4b9da0e2f19c6ac3d6ea85af27aab70677f49711a538c0ad9b5f9c2b7" exitCode=0
Oct 30 19:33:23.812336 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:23.812176    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x" event={"ID":"664f1334-638a-4581-9c84-5a2ea290ec81","Type":"ContainerDied","Data":"2eb0cdd4b9da0e2f19c6ac3d6ea85af27aab70677f49711a538c0ad9b5f9c2b7"}
Oct 30 19:33:23.812570 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:23.812555    5820 scope.go:122] "RemoveContainer" containerID="2eb0cdd4b9da0e2f19c6ac3d6ea85af27aab70677f49711a538c0ad9b5f9c2b7"
Oct 30 19:33:23.814119 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:23.814092    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:33:23.814190 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:23.814157    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:33:23.814228 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:23.814194    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:33:23.944521 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:23.944477    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:24.120678 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:24.120642    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:24.820929 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:24.820897    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x" event={"ID":"664f1334-638a-4581-9c84-5a2ea290ec81","Type":"ContainerStarted","Data":"5701229365ac15f91c11a99f8b93cb54085647dbb13efcf784fd73e507414ea0"}
Oct 30 19:33:24.859258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:24.859224    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:33:24.860418 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:24.860376    5820 patch_prober.go:29] interesting pod/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused" start-of-body=
Oct 30 19:33:24.860555 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:24.860441    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x" podUID="664f1334-638a-4581-9c84-5a2ea290ec81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused"
Oct 30 19:33:25.202792 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:25.202635    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:25.859329 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:25.859293    5820 patch_prober.go:29] interesting pod/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused" start-of-body=
Oct 30 19:33:25.859329 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:25.859343    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x" podUID="664f1334-638a-4581-9c84-5a2ea290ec81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused"
Oct 30 19:33:26.859389 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:26.859344    5820 patch_prober.go:29] interesting pod/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused" start-of-body=
Oct 30 19:33:26.860032 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:26.859997    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x" podUID="664f1334-638a-4581-9c84-5a2ea290ec81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused"
Oct 30 19:33:27.070910 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:27.070864    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:27.087586 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:27.087561    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:33:27.297396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:27.297288    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:27.861266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:27.860845    5820 patch_prober.go:29] interesting pod/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused" start-of-body=
Oct 30 19:33:27.861266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:27.860914    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x" podUID="664f1334-638a-4581-9c84-5a2ea290ec81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused"
Oct 30 19:33:28.856033 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:28.855998    5820 patch_prober.go:29] interesting pod/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused" start-of-body=
Oct 30 19:33:28.857209 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:28.857174    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x" podUID="664f1334-638a-4581-9c84-5a2ea290ec81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused"
Oct 30 19:33:29.852487 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:29.852453    5820 patch_prober.go:29] interesting pod/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused" start-of-body=
Oct 30 19:33:29.852896 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:29.852518    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x" podUID="664f1334-638a-4581-9c84-5a2ea290ec81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused"
Oct 30 19:33:30.656986 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:30.656912    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:30.857172 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:30.857144    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:33:30.858320 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:30.858299    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:33:34.257361 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:34.257328    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:35.935397 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:35.935293    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:38.204100 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:33:38.204059    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:33:41.058797 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:41.058755    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:44.858508 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:44.858471    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:45.134222 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:45.134142    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:45.355174 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:45.355138    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:45.535992 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:45.535883    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:45.978712 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:45.978677    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:50.520334 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:50.520290    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:50.747609 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:50.747573    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:55.450279 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:55.446704    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:57.071539 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:57.071496    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:57.303119 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:33:57.303083    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:00.311260 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:00.311209    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:00.499598 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:00.499557    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:00.675737 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:00.675645    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:05.294810 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:05.294762    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:06.808206 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:06.808169    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:08.204496 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:34:08.204456    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:34:10.166338 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:10.166298    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:14.933073 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:14.933039    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:19.955797 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:19.955758    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:22.957171 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:22.957134    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:24.566206 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:24.566165    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:24.934860 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:24.934693    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:25.066782 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.066749    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"]
Oct 30 19:34:25.071346 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.071324    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.142304 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.127731    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-76a1ef-openshift-default-dockercfg-h77f9\""
Oct 30 19:34:25.162009 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.158472    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv6zw\" (UniqueName: \"kubernetes.io/projected/075517b2-2a7e-4edc-8658-677e52abc0c6-kube-api-access-gv6zw\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.162009 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.158551    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-podinfo\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.162009 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.158585    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-credential-socket\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.162009 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.158625    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-envoy\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.162009 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.158653    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-token\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.162009 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.158706    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-workload-socket\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.162009 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.158796    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-data\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.162009 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.158843    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/075517b2-2a7e-4edc-8658-677e52abc0c6-istiod-ca-cert\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.162009 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.158894    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-workload-certs\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.182015 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.181974    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"]
Oct 30 19:34:25.200532 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.200457    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:25.259911 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.259872    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/075517b2-2a7e-4edc-8658-677e52abc0c6-istiod-ca-cert\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.260133 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.259929    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-workload-certs\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.260133 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.260004    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv6zw\" (UniqueName: \"kubernetes.io/projected/075517b2-2a7e-4edc-8658-677e52abc0c6-kube-api-access-gv6zw\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.260133 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.260050    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-podinfo\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.260133 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.260080    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-credential-socket\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.260133 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.260119    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-envoy\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.260435 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.260144    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-token\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.260435 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.260217    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-workload-socket\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.260435 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.260351    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-data\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.260605 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.260509    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-workload-certs\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.260734 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.260714    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/075517b2-2a7e-4edc-8658-677e52abc0c6-istiod-ca-cert\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.260794 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.260747    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-credential-socket\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.261012 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.260994    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-data\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.261221 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.261199    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-workload-socket\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.262655 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.262633    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-envoy\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.262743 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.262693    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-podinfo\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.321159 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.321134    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-token\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.321277 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.321210    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv6zw\" (UniqueName: \"kubernetes.io/projected/075517b2-2a7e-4edc-8658-677e52abc0c6-kube-api-access-gv6zw\") pod \"e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") " pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.385765 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.385737    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:25.534400 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.534309    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:25.814579 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.814554    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"]
Oct 30 19:34:25.816863 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:34:25.816833    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod075517b2_2a7e_4edc_8658_677e52abc0c6.slice/crio-35314d979835d309a6e5ae7eb2f5d916fa691e536b78f6176d5e94a52aa547f2 WatchSource:0}: Error finding container 35314d979835d309a6e5ae7eb2f5d916fa691e536b78f6176d5e94a52aa547f2: Status 404 returned error can't find the container with id 35314d979835d309a6e5ae7eb2f5d916fa691e536b78f6176d5e94a52aa547f2
Oct 30 19:34:25.819541 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.819510    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:34:25.819610 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.819575    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:34:25.819610 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:25.819603    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:34:26.245877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:26.245789    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq" event={"ID":"075517b2-2a7e-4edc-8658-677e52abc0c6","Type":"ContainerStarted","Data":"4480109f0c6042b3569cf4d0be47343829bd7b45b6b400426a26b106cde548a0"}
Oct 30 19:34:26.245877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:26.245861    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq" event={"ID":"075517b2-2a7e-4edc-8658-677e52abc0c6","Type":"ContainerStarted","Data":"35314d979835d309a6e5ae7eb2f5d916fa691e536b78f6176d5e94a52aa547f2"}
Oct 30 19:34:26.371416 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:26.371369    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq" podStartSLOduration=1.371354813 podStartE2EDuration="1.371354813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:34:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:34:26.368634278 +0000 UTC m=+3587.080382976" watchObservedRunningTime="2026-10-30 19:34:26.371354813 +0000 UTC m=+3587.083103511"
Oct 30 19:34:26.386906 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:26.386878    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:26.392671 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:26.392652    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:27.251600 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:27.251572    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:27.252585 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:27.252564    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:34:27.829382 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:27.828647    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:29.161538 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:29.161500    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:29.808974 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:29.805806    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:29.845488 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:29.845445    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"]
Oct 30 19:34:29.852857 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:29.852835    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"
Oct 30 19:34:29.890336 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:29.890311    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"]
Oct 30 19:34:29.922365 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:29.922336    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd\" (UID: \"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"
Oct 30 19:34:29.922512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:29.922401    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd\" (UID: \"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"
Oct 30 19:34:29.922590 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:29.922552    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdx7c\" (UniqueName: \"kubernetes.io/projected/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-kube-api-access-rdx7c\") pod \"maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd\" (UID: \"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"
Oct 30 19:34:29.922706 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:29.922684    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd\" (UID: \"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"
Oct 30 19:34:29.935128 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:29.935107    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w4-0f83f5-serving-cert\""
Oct 30 19:34:30.024168 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:30.024137    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd\" (UID: \"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"
Oct 30 19:34:30.024396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:30.024186    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd\" (UID: \"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"
Oct 30 19:34:30.024396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:30.024235    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdx7c\" (UniqueName: \"kubernetes.io/projected/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-kube-api-access-rdx7c\") pod \"maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd\" (UID: \"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"
Oct 30 19:34:30.024396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:30.024285    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd\" (UID: \"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"
Oct 30 19:34:30.024603 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:34:30.024446    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-serving-cert: secret "maas-api-e2e-shared-b-w4-0f83f5-serving-cert" not found
Oct 30 19:34:30.024603 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:34:30.024502    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-maas-api-tls podName:f66409e5-083e-4ae1-bc9a-02d1a4fd12b8 nodeName:}" failed. No retries permitted until 2026-10-30 19:34:30.524486215 +0000 UTC m=+3591.236234902 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-maas-api-tls") pod "maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd" (UID: "f66409e5-083e-4ae1-bc9a-02d1a4fd12b8") : secret "maas-api-e2e-shared-b-w4-0f83f5-serving-cert" not found
Oct 30 19:34:30.025011 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:30.024986    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd\" (UID: \"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"
Oct 30 19:34:30.026978 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:30.026940    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd\" (UID: \"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"
Oct 30 19:34:30.108399 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:30.108320    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdx7c\" (UniqueName: \"kubernetes.io/projected/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-kube-api-access-rdx7c\") pod \"maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd\" (UID: \"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"
Oct 30 19:34:30.123347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:30.123315    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:30.314534 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:30.314493    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:30.530509 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:30.530426    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd\" (UID: \"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"
Oct 30 19:34:30.532005 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:30.531974    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:30.533327 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:30.533306    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd\" (UID: \"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"
Oct 30 19:34:30.737709 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:30.737676    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:30.768768 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:30.768737    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"
Oct 30 19:34:30.919788 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:30.919741    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:31.180714 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:31.180689    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"]
Oct 30 19:34:31.183139 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:34:31.183112    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf66409e5_083e_4ae1_bc9a_02d1a4fd12b8.slice/crio-747e0bf42168927d0e207f62645695281615488675b9284749b1e2e4b32dead1 WatchSource:0}: Error finding container 747e0bf42168927d0e207f62645695281615488675b9284749b1e2e4b32dead1: Status 404 returned error can't find the container with id 747e0bf42168927d0e207f62645695281615488675b9284749b1e2e4b32dead1
Oct 30 19:34:31.184875 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:31.184857    5820 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:34:31.276706 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:31.276676    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd" event={"ID":"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8","Type":"ContainerStarted","Data":"747e0bf42168927d0e207f62645695281615488675b9284749b1e2e4b32dead1"}
Oct 30 19:34:32.284992 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:32.284883    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd" event={"ID":"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8","Type":"ContainerStarted","Data":"2c778d41100bb0cf27ee83370bc7ce358cfd2420bf28b6e9c5d19ab804e2c50c"}
Oct 30 19:34:32.284992 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:32.284936    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"
Oct 30 19:34:32.408015 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:32.407956    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd" podStartSLOduration=2.63872801 podStartE2EDuration="3.407927557s" totalImagesPullingTime="769.199547ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:34:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:34:32.404568118 +0000 UTC m=+3593.116316817" watchObservedRunningTime="2026-10-30 19:34:32.407927557 +0000 UTC m=+3593.119676286"
Oct 30 19:34:34.841935 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:34.841900    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:35.022940 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:35.022897    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:35.214896 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:35.214821    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:35.397816 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:35.397784    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:35.584260 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:35.584214    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:35.850215 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:35.850077    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:38.209903 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:34:38.209870    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:34:38.296823 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:38.296798    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"
Oct 30 19:34:41.065054 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:41.065018    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:41.251229 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:41.251190    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:41.439784 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:41.439695    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:46.032915 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:46.032827    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:50.719264 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:50.719161    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:50.905450 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:50.905416    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:51.105423 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:51.105381    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:53.380214 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:53.380182    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:53.627634 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:53.627595    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:55.750821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:55.750772    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:58.451564 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:58.451481    5820 generic.go:356] "Generic (PLEG): container finished" podID="c6c5724f-a1ce-448e-9535-3bbc49743b18" containerID="2fe5dcdb9e6357bd512e6ff42239be39f43fe0c0a50110c34a622d418eff7458" exitCode=0
Oct 30 19:34:58.451564 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:58.451552    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x" event={"ID":"c6c5724f-a1ce-448e-9535-3bbc49743b18","Type":"ContainerDied","Data":"2fe5dcdb9e6357bd512e6ff42239be39f43fe0c0a50110c34a622d418eff7458"}
Oct 30 19:34:58.452036 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:58.451877    5820 scope.go:122] "RemoveContainer" containerID="2fe5dcdb9e6357bd512e6ff42239be39f43fe0c0a50110c34a622d418eff7458"
Oct 30 19:34:58.453431 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:58.453401    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:34:58.453551 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:58.453471    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:34:58.453551 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:58.453501    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:34:59.465792 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:59.465760    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x" event={"ID":"c6c5724f-a1ce-448e-9535-3bbc49743b18","Type":"ContainerStarted","Data":"6cf5ae072abdb13e2fd89a61d63c0da21a75befb45b000deea95bcebf0e6e001"}
Oct 30 19:34:59.568085 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:59.568052    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:34:59.568280 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:59.568239    5820 patch_prober.go:29] interesting pod/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused" start-of-body=
Oct 30 19:34:59.568332 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:34:59.568284    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x" podUID="c6c5724f-a1ce-448e-9535-3bbc49743b18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused"
Oct 30 19:35:00.568311 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:00.568272    5820 patch_prober.go:29] interesting pod/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused" start-of-body=
Oct 30 19:35:00.568773 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:00.568328    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x" podUID="c6c5724f-a1ce-448e-9535-3bbc49743b18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused"
Oct 30 19:35:00.609977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:00.609912    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:01.567714 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:01.567681    5820 patch_prober.go:29] interesting pod/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused" start-of-body=
Oct 30 19:35:01.567961 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:01.567736    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x" podUID="c6c5724f-a1ce-448e-9535-3bbc49743b18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused"
Oct 30 19:35:02.568126 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:02.568089    5820 patch_prober.go:29] interesting pod/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused" start-of-body=
Oct 30 19:35:02.568566 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:02.568144    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x" podUID="c6c5724f-a1ce-448e-9535-3bbc49743b18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused"
Oct 30 19:35:03.568341 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:03.568305    5820 patch_prober.go:29] interesting pod/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused" start-of-body=
Oct 30 19:35:03.568763 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:03.568382    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x" podUID="c6c5724f-a1ce-448e-9535-3bbc49743b18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused"
Oct 30 19:35:04.058991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:04.057550    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:04.568176 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:04.568142    5820 patch_prober.go:29] interesting pod/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused" start-of-body=
Oct 30 19:35:04.568358 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:04.568195    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x" podUID="c6c5724f-a1ce-448e-9535-3bbc49743b18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused"
Oct 30 19:35:05.430980 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:05.428808    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:05.572385 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:05.572336    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:35:05.572814 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:05.572791    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:35:05.573564 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:05.573547    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:35:07.167722 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:07.167689    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:08.204321 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:35:08.204287    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:35:10.273940 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:10.273902    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:15.205666 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:15.205627    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:20.144251 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:20.142589    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:25.141074 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:25.141026    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:26.858638 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:26.858588    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:30.059737 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:30.059702    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:34.852715 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:34.852669    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:35.532861 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:35.532827    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:36.841363 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:36.841334    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:37.336316 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.336287    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"]
Oct 30 19:35:37.340904 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.340884    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.394219 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.394185    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-345670-openshift-default-dockercfg-4wcg7\""
Oct 30 19:35:37.447685 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.447648    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-podinfo\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.447873 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.447698    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxdbq\" (UniqueName: \"kubernetes.io/projected/a396603b-86a8-40ff-b353-0c9378d9f79f-kube-api-access-bxdbq\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.447873 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.447785    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a396603b-86a8-40ff-b353-0c9378d9f79f-istiod-ca-cert\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.448031 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.447914    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-token\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.448093 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.448025    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-workload-socket\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.448154 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.448097    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-credential-socket\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.448154 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.448125    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-envoy\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.448270 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.448242    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-data\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.448329 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.448305    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-workload-certs\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.513260 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.513222    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:37.549586 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.549550    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-podinfo\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.549985 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.549916    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxdbq\" (UniqueName: \"kubernetes.io/projected/a396603b-86a8-40ff-b353-0c9378d9f79f-kube-api-access-bxdbq\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.550194 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.549987    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a396603b-86a8-40ff-b353-0c9378d9f79f-istiod-ca-cert\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.550194 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.550120    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-token\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.550194 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.550187    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-workload-socket\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.550490 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.550261    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-credential-socket\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.550490 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.550285    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-envoy\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.550490 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.550345    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-data\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.550490 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.550421    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-workload-certs\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.550740 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.550705    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a396603b-86a8-40ff-b353-0c9378d9f79f-istiod-ca-cert\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.550740 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.550726    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-workload-certs\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.551038 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.550996    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-credential-socket\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.551262 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.551245    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-workload-socket\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.551397 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.551308    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-data\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.553047 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.553023    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-podinfo\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.553335 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.553314    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-envoy\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.554094 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.554061    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"]
Oct 30 19:35:37.616811 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.616750    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxdbq\" (UniqueName: \"kubernetes.io/projected/a396603b-86a8-40ff-b353-0c9378d9f79f-kube-api-access-bxdbq\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.617582 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.617563    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-token\") pod \"e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") " pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.654942 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.654916    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:37.755275 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:37.755147    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:38.104753 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:38.104716    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"]
Oct 30 19:35:38.136589 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:35:38.136557    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda396603b_86a8_40ff_b353_0c9378d9f79f.slice/crio-5e3ca59f14b6dfa452760ca712d68432c6d21ea1ad4ba4b4183322ecc780823c WatchSource:0}: Error finding container 5e3ca59f14b6dfa452760ca712d68432c6d21ea1ad4ba4b4183322ecc780823c: Status 404 returned error can't find the container with id 5e3ca59f14b6dfa452760ca712d68432c6d21ea1ad4ba4b4183322ecc780823c
Oct 30 19:35:38.139195 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:38.139154    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:35:38.139306 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:38.139233    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:35:38.139306 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:38.139264    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:35:38.204502 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:35:38.204469    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:35:38.709997 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:38.709938    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd" event={"ID":"a396603b-86a8-40ff-b353-0c9378d9f79f","Type":"ContainerStarted","Data":"912377f1269aab1fb9550a787c78369bd46d5da877a8bb5a52f37b33c2d3ac2a"}
Oct 30 19:35:38.710183 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:38.710012    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd" event={"ID":"a396603b-86a8-40ff-b353-0c9378d9f79f","Type":"ContainerStarted","Data":"5e3ca59f14b6dfa452760ca712d68432c6d21ea1ad4ba4b4183322ecc780823c"}
Oct 30 19:35:38.832491 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:38.832429    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd" podStartSLOduration=1.832411572 podStartE2EDuration="1.832411572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:38.829887499 +0000 UTC m=+3659.541636195" watchObservedRunningTime="2026-10-30 19:35:38.832411572 +0000 UTC m=+3659.544160270"
Oct 30 19:35:39.656126 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:39.656089    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:39.661516 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:39.661491    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:39.716053 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:39.716025    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:39.717067 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:39.717047    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:35:39.924858 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:39.924767    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:41.720245 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:41.720215    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:41.968431 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:41.968396    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:42.174349 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:42.174308    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:44.989966 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:44.989915    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:45.229709 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:45.229666    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:50.260817 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:50.260777    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:54.992709 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:54.992675    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:58.391098 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:58.391057    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:59.719013 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:59.695754    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:59.897690 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:35:59.897651    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:00.082237 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:00.082200    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:04.628670 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:04.628635    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:07.506335 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:07.506303    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:08.203232 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:36:08.203194    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:36:09.188253 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:09.188218    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"]
Oct 30 19:36:09.192648 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:09.192629    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"
Oct 30 19:36:09.250824 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:09.250787    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5f8e8dd-a036-477e-ac6e-b2f15500af33-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv\" (UID: \"a5f8e8dd-a036-477e-ac6e-b2f15500af33\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"
Oct 30 19:36:09.251034 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:09.250918    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5f8e8dd-a036-477e-ac6e-b2f15500af33-metrics-certs\") pod \"maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv\" (UID: \"a5f8e8dd-a036-477e-ac6e-b2f15500af33\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"
Oct 30 19:36:09.251034 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:09.251021    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwc49\" (UniqueName: \"kubernetes.io/projected/a5f8e8dd-a036-477e-ac6e-b2f15500af33-kube-api-access-pwc49\") pod \"maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv\" (UID: \"a5f8e8dd-a036-477e-ac6e-b2f15500af33\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"
Oct 30 19:36:09.251166 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:09.251081    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5f8e8dd-a036-477e-ac6e-b2f15500af33-maas-api-tls\") pod \"maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv\" (UID: \"a5f8e8dd-a036-477e-ac6e-b2f15500af33\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"
Oct 30 19:36:09.294737 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:09.294708    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-25e5ef-serving-cert\""
Oct 30 19:36:09.297529 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:09.297506    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"]
Oct 30 19:36:09.352821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:09.352781    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5f8e8dd-a036-477e-ac6e-b2f15500af33-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv\" (UID: \"a5f8e8dd-a036-477e-ac6e-b2f15500af33\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"
Oct 30 19:36:09.353040 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:09.352874    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5f8e8dd-a036-477e-ac6e-b2f15500af33-metrics-certs\") pod \"maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv\" (UID: \"a5f8e8dd-a036-477e-ac6e-b2f15500af33\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"
Oct 30 19:36:09.353040 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:09.352922    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwc49\" (UniqueName: \"kubernetes.io/projected/a5f8e8dd-a036-477e-ac6e-b2f15500af33-kube-api-access-pwc49\") pod \"maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv\" (UID: \"a5f8e8dd-a036-477e-ac6e-b2f15500af33\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"
Oct 30 19:36:09.353040 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:09.353011    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5f8e8dd-a036-477e-ac6e-b2f15500af33-maas-api-tls\") pod \"maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv\" (UID: \"a5f8e8dd-a036-477e-ac6e-b2f15500af33\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"
Oct 30 19:36:09.353216 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:36:09.353174    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-serving-cert: secret "maas-api-e2e-extmod-b-25e5ef-serving-cert" not found
Oct 30 19:36:09.353262 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:36:09.353241    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5f8e8dd-a036-477e-ac6e-b2f15500af33-maas-api-tls podName:a5f8e8dd-a036-477e-ac6e-b2f15500af33 nodeName:}" failed. No retries permitted until 2026-10-30 19:36:09.853224509 +0000 UTC m=+3690.564973185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a5f8e8dd-a036-477e-ac6e-b2f15500af33-maas-api-tls") pod "maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv" (UID: "a5f8e8dd-a036-477e-ac6e-b2f15500af33") : secret "maas-api-e2e-extmod-b-25e5ef-serving-cert" not found
Oct 30 19:36:09.353569 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:09.353546    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5f8e8dd-a036-477e-ac6e-b2f15500af33-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv\" (UID: \"a5f8e8dd-a036-477e-ac6e-b2f15500af33\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"
Oct 30 19:36:09.355909 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:09.355888    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5f8e8dd-a036-477e-ac6e-b2f15500af33-metrics-certs\") pod \"maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv\" (UID: \"a5f8e8dd-a036-477e-ac6e-b2f15500af33\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"
Oct 30 19:36:09.414534 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:09.414501    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwc49\" (UniqueName: \"kubernetes.io/projected/a5f8e8dd-a036-477e-ac6e-b2f15500af33-kube-api-access-pwc49\") pod \"maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv\" (UID: \"a5f8e8dd-a036-477e-ac6e-b2f15500af33\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"
Oct 30 19:36:09.434903 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:09.434874    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:09.633291 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:09.633259    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:09.833249 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:09.833211    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:09.874720 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:09.874690    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5f8e8dd-a036-477e-ac6e-b2f15500af33-maas-api-tls\") pod \"maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv\" (UID: \"a5f8e8dd-a036-477e-ac6e-b2f15500af33\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"
Oct 30 19:36:09.877363 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:09.877337    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5f8e8dd-a036-477e-ac6e-b2f15500af33-maas-api-tls\") pod \"maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv\" (UID: \"a5f8e8dd-a036-477e-ac6e-b2f15500af33\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"
Oct 30 19:36:10.106491 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:10.106459    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"
Oct 30 19:36:10.486842 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:10.486812    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"]
Oct 30 19:36:10.487647 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:36:10.487618    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5f8e8dd_a036_477e_ac6e_b2f15500af33.slice/crio-83fc05809407852b83a4b236756f9affdae30289f9fe7e8154131940cfcac60f WatchSource:0}: Error finding container 83fc05809407852b83a4b236756f9affdae30289f9fe7e8154131940cfcac60f: Status 404 returned error can't find the container with id 83fc05809407852b83a4b236756f9affdae30289f9fe7e8154131940cfcac60f
Oct 30 19:36:10.916135 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:10.916085    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv" event={"ID":"a5f8e8dd-a036-477e-ac6e-b2f15500af33","Type":"ContainerStarted","Data":"83fc05809407852b83a4b236756f9affdae30289f9fe7e8154131940cfcac60f"}
Oct 30 19:36:11.925032 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:11.924990    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv" event={"ID":"a5f8e8dd-a036-477e-ac6e-b2f15500af33","Type":"ContainerStarted","Data":"91fe75abd28985b7d8afe674574b16edb5589f9854d30b353100b7c74bd4d913"}
Oct 30 19:36:11.925495 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:11.925196    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"
Oct 30 19:36:12.045231 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:12.045170    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv" podStartSLOduration=2.035842615 podStartE2EDuration="3.045156372s" totalImagesPullingTime="1.009313757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:12.042067014 +0000 UTC m=+3692.753815713" watchObservedRunningTime="2026-10-30 19:36:12.045156372 +0000 UTC m=+3692.756905069"
Oct 30 19:36:14.730315 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:14.730274    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:17.937599 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:17.937517    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"
Oct 30 19:36:19.520753 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:19.520710    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:22.731149 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:22.727026    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:22.922461 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:22.922426    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:24.618097 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:24.618058    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:28.293138 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:28.293086    5820 patch_prober.go:29] interesting pod/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.193:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:36:28.293621 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:28.293177    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq" podUID="075517b2-2a7e-4edc-8658-677e52abc0c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.193:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:28.460284 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:28.460244    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:28.636595 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:28.636553    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:29.654365 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:29.650407    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:31.056941 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:31.056905    5820 generic.go:356] "Generic (PLEG): container finished" podID="075517b2-2a7e-4edc-8658-677e52abc0c6" containerID="4480109f0c6042b3569cf4d0be47343829bd7b45b6b400426a26b106cde548a0" exitCode=0
Oct 30 19:36:31.057413 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:31.056973    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq" event={"ID":"075517b2-2a7e-4edc-8658-677e52abc0c6","Type":"ContainerDied","Data":"4480109f0c6042b3569cf4d0be47343829bd7b45b6b400426a26b106cde548a0"}
Oct 30 19:36:31.057413 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:31.057351    5820 scope.go:122] "RemoveContainer" containerID="4480109f0c6042b3569cf4d0be47343829bd7b45b6b400426a26b106cde548a0"
Oct 30 19:36:31.058981 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:31.058935    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:36:31.059052 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:31.059030    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:36:31.059089 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:31.059065    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:36:32.067859 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:32.067824    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq" event={"ID":"075517b2-2a7e-4edc-8658-677e52abc0c6","Type":"ContainerStarted","Data":"7c83efbf546054a8c82319bf8b891609311142389067bdf1ea460866af8a8f46"}
Oct 30 19:36:32.386817 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:32.386734    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:36:32.387045 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:32.386934    5820 patch_prober.go:29] interesting pod/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused" start-of-body=
Oct 30 19:36:32.387140 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:32.387035    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq" podUID="075517b2-2a7e-4edc-8658-677e52abc0c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused"
Oct 30 19:36:33.386929 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:33.386895    5820 patch_prober.go:29] interesting pod/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused" start-of-body=
Oct 30 19:36:33.387381 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:33.386974    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq" podUID="075517b2-2a7e-4edc-8658-677e52abc0c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused"
Oct 30 19:36:34.386804 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:34.386765    5820 patch_prober.go:29] interesting pod/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused" start-of-body=
Oct 30 19:36:34.387055 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:34.386832    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq" podUID="075517b2-2a7e-4edc-8658-677e52abc0c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused"
Oct 30 19:36:34.528663 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:34.528627    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:35.386385 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:35.386351    5820 patch_prober.go:29] interesting pod/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused" start-of-body=
Oct 30 19:36:35.386564 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:35.386405    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq" podUID="075517b2-2a7e-4edc-8658-677e52abc0c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused"
Oct 30 19:36:36.386706 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:36.386669    5820 patch_prober.go:29] interesting pod/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused" start-of-body=
Oct 30 19:36:36.387153 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:36.386725    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq" podUID="075517b2-2a7e-4edc-8658-677e52abc0c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused"
Oct 30 19:36:37.049276 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:37.049240    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq"]
Oct 30 19:36:37.391854 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:37.391769    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:36:37.392282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:37.392268    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:36:37.392962 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:37.392931    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:36:37.501904 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:37.496286    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:37.761936 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:37.761854    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:38.204088 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:36:38.204049    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:36:39.663542 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:39.659923    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:42.049890 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:42.049832    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq" podUID="0a6148bf-297b-4a6a-aa29-bd4d769a4d5c" containerName="payload-processing" containerID="cri-o://fc8d6f7b72324197f2402a8215c10c84be6d9f2305800d8903c30b36e0b64c35" gracePeriod=25
Oct 30 19:36:42.309299 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:42.309241    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq"
Oct 30 19:36:42.388642 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:42.388616    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a6148bf-297b-4a6a-aa29-bd4d769a4d5c-kube-api-access-ss47b\" (UniqueName: \"kubernetes.io/projected/0a6148bf-297b-4a6a-aa29-bd4d769a4d5c-kube-api-access-ss47b\") pod \"0a6148bf-297b-4a6a-aa29-bd4d769a4d5c\" (UID: \"0a6148bf-297b-4a6a-aa29-bd4d769a4d5c\") "
Oct 30 19:36:42.388824 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:42.388805    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a6148bf-297b-4a6a-aa29-bd4d769a4d5c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a6148bf-297b-4a6a-aa29-bd4d769a4d5c-plugins-config-volume\") pod \"0a6148bf-297b-4a6a-aa29-bd4d769a4d5c\" (UID: \"0a6148bf-297b-4a6a-aa29-bd4d769a4d5c\") "
Oct 30 19:36:42.389190 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:42.389158    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a6148bf-297b-4a6a-aa29-bd4d769a4d5c-plugins-config-volume" pod "0a6148bf-297b-4a6a-aa29-bd4d769a4d5c" (UID: "0a6148bf-297b-4a6a-aa29-bd4d769a4d5c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:42.389311 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:42.389292    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a6148bf-297b-4a6a-aa29-bd4d769a4d5c-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:36:42.391042 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:42.391020    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a6148bf-297b-4a6a-aa29-bd4d769a4d5c-kube-api-access-ss47b" pod "0a6148bf-297b-4a6a-aa29-bd4d769a4d5c" (UID: "0a6148bf-297b-4a6a-aa29-bd4d769a4d5c"). InnerVolumeSpecName "kube-api-access-ss47b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:42.490438 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:42.490404    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss47b\" (UniqueName: \"kubernetes.io/projected/0a6148bf-297b-4a6a-aa29-bd4d769a4d5c-kube-api-access-ss47b\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:36:43.138020 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:43.137988    5820 generic.go:356] "Generic (PLEG): container finished" podID="0a6148bf-297b-4a6a-aa29-bd4d769a4d5c" containerID="fc8d6f7b72324197f2402a8215c10c84be6d9f2305800d8903c30b36e0b64c35" exitCode=0
Oct 30 19:36:43.138447 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:43.138046    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq"
Oct 30 19:36:43.138447 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:43.138071    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq" event={"ID":"0a6148bf-297b-4a6a-aa29-bd4d769a4d5c","Type":"ContainerDied","Data":"fc8d6f7b72324197f2402a8215c10c84be6d9f2305800d8903c30b36e0b64c35"}
Oct 30 19:36:43.138447 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:43.138151    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq" event={"ID":"0a6148bf-297b-4a6a-aa29-bd4d769a4d5c","Type":"ContainerDied","Data":"e95345f1c333ebedab3c9b12feb47fbb7cfc4be8350b4f5682b61a93946046c3"}
Oct 30 19:36:43.138447 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:43.138176    5820 scope.go:122] "RemoveContainer" containerID="fc8d6f7b72324197f2402a8215c10c84be6d9f2305800d8903c30b36e0b64c35"
Oct 30 19:36:43.149820 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:43.149801    5820 scope.go:122] "RemoveContainer" containerID="fc8d6f7b72324197f2402a8215c10c84be6d9f2305800d8903c30b36e0b64c35"
Oct 30 19:36:43.150127 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:36:43.150093    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc8d6f7b72324197f2402a8215c10c84be6d9f2305800d8903c30b36e0b64c35\": container with ID starting with fc8d6f7b72324197f2402a8215c10c84be6d9f2305800d8903c30b36e0b64c35 not found: ID does not exist" containerID="fc8d6f7b72324197f2402a8215c10c84be6d9f2305800d8903c30b36e0b64c35"
Oct 30 19:36:43.150207 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:43.150127    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc8d6f7b72324197f2402a8215c10c84be6d9f2305800d8903c30b36e0b64c35"} err="failed to get container status \"fc8d6f7b72324197f2402a8215c10c84be6d9f2305800d8903c30b36e0b64c35\": rpc error: code = NotFound desc = could not find container \"fc8d6f7b72324197f2402a8215c10c84be6d9f2305800d8903c30b36e0b64c35\": container with ID starting with fc8d6f7b72324197f2402a8215c10c84be6d9f2305800d8903c30b36e0b64c35 not found: ID does not exist"
Oct 30 19:36:43.303132 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:43.303104    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq"]
Oct 30 19:36:43.307283 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:43.307262    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-7fdf6b-7c7fbb58f8-b29hq"]
Oct 30 19:36:43.437550 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:43.437482    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a6148bf-297b-4a6a-aa29-bd4d769a4d5c" path="/var/lib/kubelet/pods/0a6148bf-297b-4a6a-aa29-bd4d769a4d5c/volumes"
Oct 30 19:36:44.550856 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:44.550822    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:49.394919 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:49.394880    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:54.324906 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:54.324873    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:59.248447 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:36:59.248405    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:04.006979 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:04.006907    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:07.856908 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:07.856864    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:08.203137 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:37:08.203063    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:37:08.860392 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:08.860345    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:13.739724 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:13.739688    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:18.233717 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.233680    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"]
Oct 30 19:37:18.234367 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.234352    5820 cpu_manager.go:414] "containerMap: removing container" podUID="0a6148bf-297b-4a6a-aa29-bd4d769a4d5c" containerName="payload-processing"
Oct 30 19:37:18.234414 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.234367    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a6148bf-297b-4a6a-aa29-bd4d769a4d5c" containerName="payload-processing"
Oct 30 19:37:18.234463 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.234453    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a6148bf-297b-4a6a-aa29-bd4d769a4d5c" containerName="payload-processing"
Oct 30 19:37:18.237932 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.237916    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"
Oct 30 19:37:18.339919 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.339890    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-345670-serving-cert\""
Oct 30 19:37:18.343210 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.343180    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"]
Oct 30 19:37:18.417347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.417313    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19f25149-4054-4b20-8674-220824e695c3-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc\" (UID: \"19f25149-4054-4b20-8674-220824e695c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"
Oct 30 19:37:18.417508 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.417445    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19f25149-4054-4b20-8674-220824e695c3-metrics-certs\") pod \"maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc\" (UID: \"19f25149-4054-4b20-8674-220824e695c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"
Oct 30 19:37:18.417565 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.417537    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19f25149-4054-4b20-8674-220824e695c3-maas-api-tls\") pod \"maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc\" (UID: \"19f25149-4054-4b20-8674-220824e695c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"
Oct 30 19:37:18.417615 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.417591    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dvf9\" (UniqueName: \"kubernetes.io/projected/19f25149-4054-4b20-8674-220824e695c3-kube-api-access-8dvf9\") pod \"maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc\" (UID: \"19f25149-4054-4b20-8674-220824e695c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"
Oct 30 19:37:18.462536 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.462505    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:18.518815 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.518743    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19f25149-4054-4b20-8674-220824e695c3-maas-api-tls\") pod \"maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc\" (UID: \"19f25149-4054-4b20-8674-220824e695c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"
Oct 30 19:37:18.518815 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.518787    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dvf9\" (UniqueName: \"kubernetes.io/projected/19f25149-4054-4b20-8674-220824e695c3-kube-api-access-8dvf9\") pod \"maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc\" (UID: \"19f25149-4054-4b20-8674-220824e695c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"
Oct 30 19:37:18.519100 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.518868    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19f25149-4054-4b20-8674-220824e695c3-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc\" (UID: \"19f25149-4054-4b20-8674-220824e695c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"
Oct 30 19:37:18.519100 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.518991    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19f25149-4054-4b20-8674-220824e695c3-metrics-certs\") pod \"maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc\" (UID: \"19f25149-4054-4b20-8674-220824e695c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"
Oct 30 19:37:18.519665 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.519638    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19f25149-4054-4b20-8674-220824e695c3-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc\" (UID: \"19f25149-4054-4b20-8674-220824e695c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"
Oct 30 19:37:18.521511 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.521489    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19f25149-4054-4b20-8674-220824e695c3-maas-api-tls\") pod \"maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc\" (UID: \"19f25149-4054-4b20-8674-220824e695c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"
Oct 30 19:37:18.521619 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.521587    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19f25149-4054-4b20-8674-220824e695c3-metrics-certs\") pod \"maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc\" (UID: \"19f25149-4054-4b20-8674-220824e695c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"
Oct 30 19:37:18.580719 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.580695    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dvf9\" (UniqueName: \"kubernetes.io/projected/19f25149-4054-4b20-8674-220824e695c3-kube-api-access-8dvf9\") pod \"maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc\" (UID: \"19f25149-4054-4b20-8674-220824e695c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"
Oct 30 19:37:18.653615 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.653176    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:18.852287 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.852251    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"
Oct 30 19:37:18.865933 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:18.865898    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:19.233711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:19.233689    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"]
Oct 30 19:37:19.235127 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:37:19.235095    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19f25149_4054_4b20_8674_220824e695c3.slice/crio-ac3c3702cd291fb22ee77ad626f666902b23c881427d6951a32e05e4bdf4f720 WatchSource:0}: Error finding container ac3c3702cd291fb22ee77ad626f666902b23c881427d6951a32e05e4bdf4f720: Status 404 returned error can't find the container with id ac3c3702cd291fb22ee77ad626f666902b23c881427d6951a32e05e4bdf4f720
Oct 30 19:37:19.388058 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:19.388024    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc" event={"ID":"19f25149-4054-4b20-8674-220824e695c3","Type":"ContainerStarted","Data":"ac3c3702cd291fb22ee77ad626f666902b23c881427d6951a32e05e4bdf4f720"}
Oct 30 19:37:20.397061 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:20.396933    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc" event={"ID":"19f25149-4054-4b20-8674-220824e695c3","Type":"ContainerStarted","Data":"6cb55ff0679e093579b0af7c9f2b8fe2f884f0a270ead55c07e6be2ac023dc2a"}
Oct 30 19:37:20.397061 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:20.397052    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"
Oct 30 19:37:20.519028 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:20.518977    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc" podStartSLOduration=1.6402853290000001 podStartE2EDuration="2.518962646s" totalImagesPullingTime="878.677317ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:20.515090522 +0000 UTC m=+3761.226839219" watchObservedRunningTime="2026-10-30 19:37:20.518962646 +0000 UTC m=+3761.230711389"
Oct 30 19:37:23.425982 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:23.425912    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:26.408825 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:26.408793    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"
Oct 30 19:37:27.570261 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:27.570227    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:29.261748 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:29.261709    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:29.601347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:29.601312    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:29.790837 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:29.790808    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:30.507977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:30.507911    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:34.786608 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:34.786497    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:37.420375 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:37.420329    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:37.673279 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:37.673191    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:37.902940 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:37.902902    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:38.204180 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:37:38.204141    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:37:39.556127 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:39.556087    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:40.218396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:40.218355    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:40.522785 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:40.522694    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:41.640772 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:41.640733    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:44.806566 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:44.806532    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:45.028575 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:45.028535    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:46.954997 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:46.954021    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:49.343672 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:49.343637    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:49.606073 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:49.605987    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:50.268819 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:50.268755    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:51.192440 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.192401    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"]
Oct 30 19:37:51.193321 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.193267    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9" podUID="af249134-3666-43cf-b772-8178f036a60d" containerName="maas-api" containerID="cri-o://7fb52710100ff11d538ea514e078a5da6a96959c09275e384c661f6e64bdc1ca" gracePeriod=30
Oct 30 19:37:51.206365 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.206329    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9" podUID="af249134-3666-43cf-b772-8178f036a60d" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.190:8443/health\": dial tcp 10.134.0.190:8443: connect: connection refused"
Oct 30 19:37:51.344846 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.344801    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:51.525176 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.525150    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"
Oct 30 19:37:51.633216 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.633172    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/af249134-3666-43cf-b772-8178f036a60d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af249134-3666-43cf-b772-8178f036a60d-maas-api-tls\") pod \"af249134-3666-43cf-b772-8178f036a60d\" (UID: \"af249134-3666-43cf-b772-8178f036a60d\") "
Oct 30 19:37:51.633457 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.633398    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/af249134-3666-43cf-b772-8178f036a60d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/af249134-3666-43cf-b772-8178f036a60d-metrics-certs\") pod \"af249134-3666-43cf-b772-8178f036a60d\" (UID: \"af249134-3666-43cf-b772-8178f036a60d\") "
Oct 30 19:37:51.633943 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.633913    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af249134-3666-43cf-b772-8178f036a60d-kube-api-access-hx2cp\" (UniqueName: \"kubernetes.io/projected/af249134-3666-43cf-b772-8178f036a60d-kube-api-access-hx2cp\") pod \"af249134-3666-43cf-b772-8178f036a60d\" (UID: \"af249134-3666-43cf-b772-8178f036a60d\") "
Oct 30 19:37:51.634131 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.634057    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af249134-3666-43cf-b772-8178f036a60d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af249134-3666-43cf-b772-8178f036a60d-openshift-service-ca\") pod \"af249134-3666-43cf-b772-8178f036a60d\" (UID: \"af249134-3666-43cf-b772-8178f036a60d\") "
Oct 30 19:37:51.635339 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.635310    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af249134-3666-43cf-b772-8178f036a60d-openshift-service-ca" pod "af249134-3666-43cf-b772-8178f036a60d" (UID: "af249134-3666-43cf-b772-8178f036a60d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:37:51.636399 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.636359    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af249134-3666-43cf-b772-8178f036a60d-metrics-certs" pod "af249134-3666-43cf-b772-8178f036a60d" (UID: "af249134-3666-43cf-b772-8178f036a60d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:37:51.636512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.636390    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af249134-3666-43cf-b772-8178f036a60d-maas-api-tls" pod "af249134-3666-43cf-b772-8178f036a60d" (UID: "af249134-3666-43cf-b772-8178f036a60d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:37:51.637323 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.637304    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af249134-3666-43cf-b772-8178f036a60d-kube-api-access-hx2cp" pod "af249134-3666-43cf-b772-8178f036a60d" (UID: "af249134-3666-43cf-b772-8178f036a60d"). InnerVolumeSpecName "kube-api-access-hx2cp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:37:51.687636 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.687605    5820 generic.go:356] "Generic (PLEG): container finished" podID="af249134-3666-43cf-b772-8178f036a60d" containerID="7fb52710100ff11d538ea514e078a5da6a96959c09275e384c661f6e64bdc1ca" exitCode=0
Oct 30 19:37:51.687834 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.687702    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"
Oct 30 19:37:51.687834 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.687736    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9" event={"ID":"af249134-3666-43cf-b772-8178f036a60d","Type":"ContainerDied","Data":"7fb52710100ff11d538ea514e078a5da6a96959c09275e384c661f6e64bdc1ca"}
Oct 30 19:37:51.687834 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.687800    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9" event={"ID":"af249134-3666-43cf-b772-8178f036a60d","Type":"ContainerDied","Data":"0022e4ea90c76c9985855205c2e66dab841ec74d31c1380ab86cc3bffa6d9557"}
Oct 30 19:37:51.687834 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.687827    5820 scope.go:122] "RemoveContainer" containerID="7fb52710100ff11d538ea514e078a5da6a96959c09275e384c661f6e64bdc1ca"
Oct 30 19:37:51.699598 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.699434    5820 scope.go:122] "RemoveContainer" containerID="7fb52710100ff11d538ea514e078a5da6a96959c09275e384c661f6e64bdc1ca"
Oct 30 19:37:51.699927 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:37:51.699899    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fb52710100ff11d538ea514e078a5da6a96959c09275e384c661f6e64bdc1ca\": container with ID starting with 7fb52710100ff11d538ea514e078a5da6a96959c09275e384c661f6e64bdc1ca not found: ID does not exist" containerID="7fb52710100ff11d538ea514e078a5da6a96959c09275e384c661f6e64bdc1ca"
Oct 30 19:37:51.700063 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.699936    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fb52710100ff11d538ea514e078a5da6a96959c09275e384c661f6e64bdc1ca"} err="failed to get container status \"7fb52710100ff11d538ea514e078a5da6a96959c09275e384c661f6e64bdc1ca\": rpc error: code = NotFound desc = could not find container \"7fb52710100ff11d538ea514e078a5da6a96959c09275e384c661f6e64bdc1ca\": container with ID starting with 7fb52710100ff11d538ea514e078a5da6a96959c09275e384c661f6e64bdc1ca not found: ID does not exist"
Oct 30 19:37:51.736379 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.736296    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx2cp\" (UniqueName: \"kubernetes.io/projected/af249134-3666-43cf-b772-8178f036a60d-kube-api-access-hx2cp\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:51.736379 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.736323    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af249134-3666-43cf-b772-8178f036a60d-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:51.736379 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.736333    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af249134-3666-43cf-b772-8178f036a60d-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:51.736379 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.736342    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af249134-3666-43cf-b772-8178f036a60d-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:51.856072 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.856031    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"]
Oct 30 19:37:51.860454 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:51.860428    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-7fdf6b-bd4df7c87-chxn9"]
Oct 30 19:37:52.983350 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:52.983312    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:53.242866 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:53.242784    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:53.425978 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:53.425926    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:53.439163 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:53.439136    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af249134-3666-43cf-b772-8178f036a60d" path="/var/lib/kubelet/pods/af249134-3666-43cf-b772-8178f036a60d/volumes"
Oct 30 19:37:54.012468 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:54.012427    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:54.209191 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:54.207428    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:54.391456 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:54.391414    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:55.044664 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:55.043197    5820 patch_prober.go:29] interesting pod/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.195:15021/healthz/ready\": read tcp 10.134.0.2:35428->10.134.0.195:15021: read: connection reset by peer" start-of-body=
Oct 30 19:37:55.044664 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:55.043300    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd" podUID="a396603b-86a8-40ff-b353-0c9378d9f79f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.195:15021/healthz/ready\": read tcp 10.134.0.2:35428->10.134.0.195:15021: read: connection reset by peer"
Oct 30 19:37:55.732489 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:55.732452    5820 generic.go:356] "Generic (PLEG): container finished" podID="a396603b-86a8-40ff-b353-0c9378d9f79f" containerID="912377f1269aab1fb9550a787c78369bd46d5da877a8bb5a52f37b33c2d3ac2a" exitCode=0
Oct 30 19:37:55.732692 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:55.732509    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd" event={"ID":"a396603b-86a8-40ff-b353-0c9378d9f79f","Type":"ContainerDied","Data":"912377f1269aab1fb9550a787c78369bd46d5da877a8bb5a52f37b33c2d3ac2a"}
Oct 30 19:37:55.733276 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:55.733250    5820 scope.go:122] "RemoveContainer" containerID="912377f1269aab1fb9550a787c78369bd46d5da877a8bb5a52f37b33c2d3ac2a"
Oct 30 19:37:55.737444 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:55.737409    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:37:55.737580 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:55.737524    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:37:55.737651 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:55.737576    5820 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 19:37:56.772232 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:56.772192    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd" event={"ID":"a396603b-86a8-40ff-b353-0c9378d9f79f","Type":"ContainerStarted","Data":"60ab157a32c0fea71740d1088845ee8b716728df1f2303385e6f945d9d09576a"}
Oct 30 19:37:56.966180 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:56.966141    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:57.196673 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:57.195696    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:57.211391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:57.211361    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"]
Oct 30 19:37:57.211597 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:57.211575    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq" podUID="075517b2-2a7e-4edc-8658-677e52abc0c6" containerName="istio-proxy" containerID="cri-o://7c83efbf546054a8c82319bf8b891609311142389067bdf1ea460866af8a8f46" gracePeriod=30
Oct 30 19:37:57.417078 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:57.417039    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:57.659049 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:57.655248    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:37:57.659049 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:57.658042    5820 patch_prober.go:29] interesting pod/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused" start-of-body=
Oct 30 19:37:57.659049 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:57.658098    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd" podUID="a396603b-86a8-40ff-b353-0c9378d9f79f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused"
Oct 30 19:37:58.613663 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:58.613630    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:58.658976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:58.657311    5820 patch_prober.go:29] interesting pod/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused" start-of-body=
Oct 30 19:37:58.658976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:58.657387    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd" podUID="a396603b-86a8-40ff-b353-0c9378d9f79f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused"
Oct 30 19:37:59.488121 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:59.488072    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:59.659468 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:59.659429    5820 patch_prober.go:29] interesting pod/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused" start-of-body=
Oct 30 19:37:59.659882 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:37:59.659495    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd" podUID="a396603b-86a8-40ff-b353-0c9378d9f79f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused"
Oct 30 19:38:00.659398 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:00.659356    5820 patch_prober.go:29] interesting pod/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused" start-of-body=
Oct 30 19:38:00.659592 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:00.659414    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd" podUID="a396603b-86a8-40ff-b353-0c9378d9f79f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused"
Oct 30 19:38:01.655383 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:01.655337    5820 patch_prober.go:29] interesting pod/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused" start-of-body=
Oct 30 19:38:01.655703 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:01.655671    5820 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd" podUID="a396603b-86a8-40ff-b353-0c9378d9f79f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused"
Oct 30 19:38:02.009814 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.009732    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:02.205200 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.205164    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:02.425178 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.424867    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:02.534641 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.534612    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:38:02.635581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.635501    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-workload-socket\") pod \"075517b2-2a7e-4edc-8658-677e52abc0c6\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") "
Oct 30 19:38:02.635581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.635541    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-credential-socket\") pod \"075517b2-2a7e-4edc-8658-677e52abc0c6\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") "
Oct 30 19:38:02.635821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.635593    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-workload-certs\") pod \"075517b2-2a7e-4edc-8658-677e52abc0c6\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") "
Oct 30 19:38:02.635821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.635612    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/075517b2-2a7e-4edc-8658-677e52abc0c6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/075517b2-2a7e-4edc-8658-677e52abc0c6-istiod-ca-cert\") pod \"075517b2-2a7e-4edc-8658-677e52abc0c6\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") "
Oct 30 19:38:02.635821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.635645    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-envoy\") pod \"075517b2-2a7e-4edc-8658-677e52abc0c6\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") "
Oct 30 19:38:02.635821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.635704    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-token\" (UniqueName: \"kubernetes.io/projected/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-token\") pod \"075517b2-2a7e-4edc-8658-677e52abc0c6\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") "
Oct 30 19:38:02.635821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.635805    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-podinfo\") pod \"075517b2-2a7e-4edc-8658-677e52abc0c6\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") "
Oct 30 19:38:02.636123 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.635846    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-data\") pod \"075517b2-2a7e-4edc-8658-677e52abc0c6\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") "
Oct 30 19:38:02.636123 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.635871    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-workload-certs" pod "075517b2-2a7e-4edc-8658-677e52abc0c6" (UID: "075517b2-2a7e-4edc-8658-677e52abc0c6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:02.636123 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.635883    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-credential-socket" pod "075517b2-2a7e-4edc-8658-677e52abc0c6" (UID: "075517b2-2a7e-4edc-8658-677e52abc0c6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:02.636123 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.635882    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/075517b2-2a7e-4edc-8658-677e52abc0c6-kube-api-access-gv6zw\" (UniqueName: \"kubernetes.io/projected/075517b2-2a7e-4edc-8658-677e52abc0c6-kube-api-access-gv6zw\") pod \"075517b2-2a7e-4edc-8658-677e52abc0c6\" (UID: \"075517b2-2a7e-4edc-8658-677e52abc0c6\") "
Oct 30 19:38:02.636123 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.636028    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-workload-socket" pod "075517b2-2a7e-4edc-8658-677e52abc0c6" (UID: "075517b2-2a7e-4edc-8658-677e52abc0c6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:02.636463 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.636179    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/075517b2-2a7e-4edc-8658-677e52abc0c6-istiod-ca-cert" pod "075517b2-2a7e-4edc-8658-677e52abc0c6" (UID: "075517b2-2a7e-4edc-8658-677e52abc0c6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:02.636463 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.636213    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-data" pod "075517b2-2a7e-4edc-8658-677e52abc0c6" (UID: "075517b2-2a7e-4edc-8658-677e52abc0c6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:02.636554 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.636520    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:02.636554 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.636542    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:02.636632 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.636557    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:02.636632 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.636571    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:02.636632 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.636585    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/075517b2-2a7e-4edc-8658-677e52abc0c6-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:02.638736 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.638697    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-podinfo" pod "075517b2-2a7e-4edc-8658-677e52abc0c6" (UID: "075517b2-2a7e-4edc-8658-677e52abc0c6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:38:02.638860 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.638747    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-token" pod "075517b2-2a7e-4edc-8658-677e52abc0c6" (UID: "075517b2-2a7e-4edc-8658-677e52abc0c6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:02.638860 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.638780    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-envoy" pod "075517b2-2a7e-4edc-8658-677e52abc0c6" (UID: "075517b2-2a7e-4edc-8658-677e52abc0c6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:02.638860 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.638787    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/075517b2-2a7e-4edc-8658-677e52abc0c6-kube-api-access-gv6zw" pod "075517b2-2a7e-4edc-8658-677e52abc0c6" (UID: "075517b2-2a7e-4edc-8658-677e52abc0c6"). InnerVolumeSpecName "kube-api-access-gv6zw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:02.659893 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.659868    5820 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:38:02.660333 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.660311    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:38:02.661225 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.661205    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:38:02.737829 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.737783    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:02.737829 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.737818    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:02.737829 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.737835    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/075517b2-2a7e-4edc-8658-677e52abc0c6-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:02.738141 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.737851    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gv6zw\" (UniqueName: \"kubernetes.io/projected/075517b2-2a7e-4edc-8658-677e52abc0c6-kube-api-access-gv6zw\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:02.814617 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.814575    5820 generic.go:356] "Generic (PLEG): container finished" podID="075517b2-2a7e-4edc-8658-677e52abc0c6" containerID="7c83efbf546054a8c82319bf8b891609311142389067bdf1ea460866af8a8f46" exitCode=0
Oct 30 19:38:02.814821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.814620    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"
Oct 30 19:38:02.814821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.814661    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq" event={"ID":"075517b2-2a7e-4edc-8658-677e52abc0c6","Type":"ContainerDied","Data":"7c83efbf546054a8c82319bf8b891609311142389067bdf1ea460866af8a8f46"}
Oct 30 19:38:02.814821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.814714    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq" event={"ID":"075517b2-2a7e-4edc-8658-677e52abc0c6","Type":"ContainerDied","Data":"35314d979835d309a6e5ae7eb2f5d916fa691e536b78f6176d5e94a52aa547f2"}
Oct 30 19:38:02.814821 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.814738    5820 scope.go:122] "RemoveContainer" containerID="7c83efbf546054a8c82319bf8b891609311142389067bdf1ea460866af8a8f46"
Oct 30 19:38:02.827872 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.827853    5820 scope.go:122] "RemoveContainer" containerID="4480109f0c6042b3569cf4d0be47343829bd7b45b6b400426a26b106cde548a0"
Oct 30 19:38:02.845407 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.845383    5820 scope.go:122] "RemoveContainer" containerID="7c83efbf546054a8c82319bf8b891609311142389067bdf1ea460866af8a8f46"
Oct 30 19:38:02.845683 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:38:02.845659    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c83efbf546054a8c82319bf8b891609311142389067bdf1ea460866af8a8f46\": container with ID starting with 7c83efbf546054a8c82319bf8b891609311142389067bdf1ea460866af8a8f46 not found: ID does not exist" containerID="7c83efbf546054a8c82319bf8b891609311142389067bdf1ea460866af8a8f46"
Oct 30 19:38:02.845759 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.845697    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c83efbf546054a8c82319bf8b891609311142389067bdf1ea460866af8a8f46"} err="failed to get container status \"7c83efbf546054a8c82319bf8b891609311142389067bdf1ea460866af8a8f46\": rpc error: code = NotFound desc = could not find container \"7c83efbf546054a8c82319bf8b891609311142389067bdf1ea460866af8a8f46\": container with ID starting with 7c83efbf546054a8c82319bf8b891609311142389067bdf1ea460866af8a8f46 not found: ID does not exist"
Oct 30 19:38:02.845759 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.845718    5820 scope.go:122] "RemoveContainer" containerID="4480109f0c6042b3569cf4d0be47343829bd7b45b6b400426a26b106cde548a0"
Oct 30 19:38:02.846039 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:38:02.846022    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4480109f0c6042b3569cf4d0be47343829bd7b45b6b400426a26b106cde548a0\": container with ID starting with 4480109f0c6042b3569cf4d0be47343829bd7b45b6b400426a26b106cde548a0 not found: ID does not exist" containerID="4480109f0c6042b3569cf4d0be47343829bd7b45b6b400426a26b106cde548a0"
Oct 30 19:38:02.846097 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:02.846048    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4480109f0c6042b3569cf4d0be47343829bd7b45b6b400426a26b106cde548a0"} err="failed to get container status \"4480109f0c6042b3569cf4d0be47343829bd7b45b6b400426a26b106cde548a0\": rpc error: code = NotFound desc = could not find container \"4480109f0c6042b3569cf4d0be47343829bd7b45b6b400426a26b106cde548a0\": container with ID starting with 4480109f0c6042b3569cf4d0be47343829bd7b45b6b400426a26b106cde548a0 not found: ID does not exist"
Oct 30 19:38:03.028433 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:03.028343    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"]
Oct 30 19:38:03.030793 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:03.030773    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-76a1ef-openshift-default-6bbbb79985-nt7nq"]
Oct 30 19:38:03.438828 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:03.438797    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="075517b2-2a7e-4edc-8658-677e52abc0c6" path="/var/lib/kubelet/pods/075517b2-2a7e-4edc-8658-677e52abc0c6/volumes"
Oct 30 19:38:04.050016 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:04.047837    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:04.566208 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:04.566154    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:08.199063 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:08.199027    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:08.204568 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:38:08.204536    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:38:09.148386 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:09.148340    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:12.013282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:12.013237    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:12.255607 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:12.255573    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:14.459133 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:14.459093    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:14.663213 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:14.663171    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:14.871125 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:14.871085    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:19.352452 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:19.352413    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:24.165323 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:24.165273    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:29.166102 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:29.166063    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:33.929509 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:33.929470    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:38.204010 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:38:38.203941    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:38:38.827922 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:38.827884    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:42.433264 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:42.433226    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:42.685718 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:42.685630    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:44.352851 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:44.352802    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:44.591203 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:44.591142    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:49.464747 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:49.464708    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:50.898504 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:50.898467    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:51.170348 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:51.170264    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:53.114865 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:53.114698    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:53.311121 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:53.311084    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:54.542888 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:54.542849    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:59.308371 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:59.308332    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"]
Oct 30 19:38:59.309110 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:59.308568    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26" podUID="1ad9d9e9-639f-49a3-a53b-e0dda71e87b4" containerName="maas-api" containerID="cri-o://5eb73dbb33cd12028ccedbcea8b9fe8c6deb7e3156f9e0eb4ec97c35b103602a" gracePeriod=30
Oct 30 19:38:59.466864 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:59.466828    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:59.601234 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:59.601210    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"
Oct 30 19:38:59.661437 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:59.661403    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-metrics-certs\") pod \"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4\" (UID: \"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4\") "
Oct 30 19:38:59.661593 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:59.661441    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-kube-api-access-pkldj\" (UniqueName: \"kubernetes.io/projected/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-kube-api-access-pkldj\") pod \"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4\" (UID: \"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4\") "
Oct 30 19:38:59.661593 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:59.661539    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-openshift-service-ca\") pod \"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4\" (UID: \"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4\") "
Oct 30 19:38:59.661593 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:59.661576    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-maas-api-tls\") pod \"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4\" (UID: \"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4\") "
Oct 30 19:38:59.662030 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:59.661926    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-openshift-service-ca" pod "1ad9d9e9-639f-49a3-a53b-e0dda71e87b4" (UID: "1ad9d9e9-639f-49a3-a53b-e0dda71e87b4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:59.663866 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:59.663833    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-metrics-certs" pod "1ad9d9e9-639f-49a3-a53b-e0dda71e87b4" (UID: "1ad9d9e9-639f-49a3-a53b-e0dda71e87b4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:38:59.664021 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:59.663868    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-kube-api-access-pkldj" pod "1ad9d9e9-639f-49a3-a53b-e0dda71e87b4" (UID: "1ad9d9e9-639f-49a3-a53b-e0dda71e87b4"). InnerVolumeSpecName "kube-api-access-pkldj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:59.664021 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:59.663929    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-maas-api-tls" pod "1ad9d9e9-639f-49a3-a53b-e0dda71e87b4" (UID: "1ad9d9e9-639f-49a3-a53b-e0dda71e87b4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:38:59.762711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:59.762680    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:59.762711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:59.762712    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:59.762929 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:59.762721    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:59.762929 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:38:59.762733    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkldj\" (UniqueName: \"kubernetes.io/projected/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4-kube-api-access-pkldj\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:00.226013 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:00.225940    5820 generic.go:356] "Generic (PLEG): container finished" podID="1ad9d9e9-639f-49a3-a53b-e0dda71e87b4" containerID="5eb73dbb33cd12028ccedbcea8b9fe8c6deb7e3156f9e0eb4ec97c35b103602a" exitCode=0
Oct 30 19:39:00.226013 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:00.226000    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"
Oct 30 19:39:00.226282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:00.226052    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26" event={"ID":"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4","Type":"ContainerDied","Data":"5eb73dbb33cd12028ccedbcea8b9fe8c6deb7e3156f9e0eb4ec97c35b103602a"}
Oct 30 19:39:00.226282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:00.226101    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26" event={"ID":"1ad9d9e9-639f-49a3-a53b-e0dda71e87b4","Type":"ContainerDied","Data":"50d9d4efff6a94082af3b07f6ae7988ea7ea126beab5d6d3b3a6dd49c606ef92"}
Oct 30 19:39:00.226282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:00.226125    5820 scope.go:122] "RemoveContainer" containerID="5eb73dbb33cd12028ccedbcea8b9fe8c6deb7e3156f9e0eb4ec97c35b103602a"
Oct 30 19:39:00.236840 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:00.236821    5820 scope.go:122] "RemoveContainer" containerID="5eb73dbb33cd12028ccedbcea8b9fe8c6deb7e3156f9e0eb4ec97c35b103602a"
Oct 30 19:39:00.237126 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:39:00.237108    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eb73dbb33cd12028ccedbcea8b9fe8c6deb7e3156f9e0eb4ec97c35b103602a\": container with ID starting with 5eb73dbb33cd12028ccedbcea8b9fe8c6deb7e3156f9e0eb4ec97c35b103602a not found: ID does not exist" containerID="5eb73dbb33cd12028ccedbcea8b9fe8c6deb7e3156f9e0eb4ec97c35b103602a"
Oct 30 19:39:00.237185 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:00.237134    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eb73dbb33cd12028ccedbcea8b9fe8c6deb7e3156f9e0eb4ec97c35b103602a"} err="failed to get container status \"5eb73dbb33cd12028ccedbcea8b9fe8c6deb7e3156f9e0eb4ec97c35b103602a\": rpc error: code = NotFound desc = could not find container \"5eb73dbb33cd12028ccedbcea8b9fe8c6deb7e3156f9e0eb4ec97c35b103602a\": container with ID starting with 5eb73dbb33cd12028ccedbcea8b9fe8c6deb7e3156f9e0eb4ec97c35b103602a not found: ID does not exist"
Oct 30 19:39:00.400566 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:00.400528    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"]
Oct 30 19:39:00.449602 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:00.449563    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-cdd844-5849b7bb7-5sl26"]
Oct 30 19:39:01.166810 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:01.166777    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:01.437848 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:01.437769    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1ad9d9e9-639f-49a3-a53b-e0dda71e87b4" path="/var/lib/kubelet/pods/1ad9d9e9-639f-49a3-a53b-e0dda71e87b4/volumes"
Oct 30 19:39:01.471941 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:01.469859    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:01.661450 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:01.661403    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:02.123343 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:02.123240    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:06.006869 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:06.006835    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"]
Oct 30 19:39:06.007399 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:06.007080    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd" podUID="f66409e5-083e-4ae1-bc9a-02d1a4fd12b8" containerName="maas-api" containerID="cri-o://2c778d41100bb0cf27ee83370bc7ce358cfd2420bf28b6e9c5d19ab804e2c50c" gracePeriod=30
Oct 30 19:39:06.136749 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:06.136715    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:06.273055 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:06.273021    5820 generic.go:356] "Generic (PLEG): container finished" podID="f66409e5-083e-4ae1-bc9a-02d1a4fd12b8" containerID="2c778d41100bb0cf27ee83370bc7ce358cfd2420bf28b6e9c5d19ab804e2c50c" exitCode=0
Oct 30 19:39:06.273196 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:06.273096    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd" event={"ID":"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8","Type":"ContainerDied","Data":"2c778d41100bb0cf27ee83370bc7ce358cfd2420bf28b6e9c5d19ab804e2c50c"}
Oct 30 19:39:06.297074 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:06.297053    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"
Oct 30 19:39:06.344912 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:06.344882    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-maas-api-tls\") pod \"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8\" (UID: \"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8\") "
Oct 30 19:39:06.345083 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:06.345035    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-openshift-service-ca\") pod \"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8\" (UID: \"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8\") "
Oct 30 19:39:06.345083 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:06.345069    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-metrics-certs\") pod \"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8\" (UID: \"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8\") "
Oct 30 19:39:06.345225 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:06.345107    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-kube-api-access-rdx7c\" (UniqueName: \"kubernetes.io/projected/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-kube-api-access-rdx7c\") pod \"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8\" (UID: \"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8\") "
Oct 30 19:39:06.345402 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:06.345374    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-openshift-service-ca" pod "f66409e5-083e-4ae1-bc9a-02d1a4fd12b8" (UID: "f66409e5-083e-4ae1-bc9a-02d1a4fd12b8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:39:06.345614 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:06.345597    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:06.347312 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:06.347283    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-maas-api-tls" pod "f66409e5-083e-4ae1-bc9a-02d1a4fd12b8" (UID: "f66409e5-083e-4ae1-bc9a-02d1a4fd12b8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:39:06.347445 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:06.347346    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-metrics-certs" pod "f66409e5-083e-4ae1-bc9a-02d1a4fd12b8" (UID: "f66409e5-083e-4ae1-bc9a-02d1a4fd12b8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:39:06.347445 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:06.347385    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-kube-api-access-rdx7c" pod "f66409e5-083e-4ae1-bc9a-02d1a4fd12b8" (UID: "f66409e5-083e-4ae1-bc9a-02d1a4fd12b8"). InnerVolumeSpecName "kube-api-access-rdx7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:39:06.447325 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:06.447289    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:06.447325 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:06.447318    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdx7c\" (UniqueName: \"kubernetes.io/projected/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-kube-api-access-rdx7c\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:06.447325 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:06.447329    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:07.280604 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:07.280565    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd" event={"ID":"f66409e5-083e-4ae1-bc9a-02d1a4fd12b8","Type":"ContainerDied","Data":"747e0bf42168927d0e207f62645695281615488675b9284749b1e2e4b32dead1"}
Oct 30 19:39:07.281125 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:07.280630    5820 scope.go:122] "RemoveContainer" containerID="2c778d41100bb0cf27ee83370bc7ce358cfd2420bf28b6e9c5d19ab804e2c50c"
Oct 30 19:39:07.281125 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:07.280586    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"
Oct 30 19:39:07.456076 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:07.456044    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"]
Oct 30 19:39:07.463525 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:07.463501    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-0f83f5-746bdbf6b7-tl2dd"]
Oct 30 19:39:07.743714 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:07.743682    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:07.982745 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:07.982713    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:08.180172 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:08.180136    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:08.203764 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:39:08.203731    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:39:08.594503 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:08.594469    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:08.815640 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:08.815602    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:09.437419 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:09.437387    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f66409e5-083e-4ae1-bc9a-02d1a4fd12b8" path="/var/lib/kubelet/pods/f66409e5-083e-4ae1-bc9a-02d1a4fd12b8/volumes"
Oct 30 19:39:10.133300 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:10.133259    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:10.330560 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:10.330524    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:10.524407 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:10.524319    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"]
Oct 30 19:39:10.524575 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:10.524550    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x" podUID="664f1334-638a-4581-9c84-5a2ea290ec81" containerName="istio-proxy" containerID="cri-o://5701229365ac15f91c11a99f8b93cb54085647dbb13efcf784fd73e507414ea0" gracePeriod=30
Oct 30 19:39:10.561730 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:10.561696    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:13.477088 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:13.477049    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:15.595964 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:15.595910    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:15.803393 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:15.803370    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:39:15.828674 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:15.828629    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:15.907718 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:15.907682    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"]
Oct 30 19:39:15.907919 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:15.907900    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x" podUID="c6c5724f-a1ce-448e-9535-3bbc49743b18" containerName="istio-proxy" containerID="cri-o://6cf5ae072abdb13e2fd89a61d63c0da21a75befb45b000deea95bcebf0e6e001" gracePeriod=30
Oct 30 19:39:15.989343 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:15.989293    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/664f1334-638a-4581-9c84-5a2ea290ec81-kube-api-access-85bsq\" (UniqueName: \"kubernetes.io/projected/664f1334-638a-4581-9c84-5a2ea290ec81-kube-api-access-85bsq\") pod \"664f1334-638a-4581-9c84-5a2ea290ec81\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") "
Oct 30 19:39:15.989553 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:15.989361    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-istio-envoy\") pod \"664f1334-638a-4581-9c84-5a2ea290ec81\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") "
Oct 30 19:39:15.989553 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:15.989421    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-workload-socket\") pod \"664f1334-638a-4581-9c84-5a2ea290ec81\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") "
Oct 30 19:39:15.989553 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:15.989505    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/664f1334-638a-4581-9c84-5a2ea290ec81-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/664f1334-638a-4581-9c84-5a2ea290ec81-istio-podinfo\") pod \"664f1334-638a-4581-9c84-5a2ea290ec81\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") "
Oct 30 19:39:15.989553 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:15.989548    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/664f1334-638a-4581-9c84-5a2ea290ec81-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/664f1334-638a-4581-9c84-5a2ea290ec81-istiod-ca-cert\") pod \"664f1334-638a-4581-9c84-5a2ea290ec81\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") "
Oct 30 19:39:15.989769 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:15.989710    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-workload-certs\") pod \"664f1334-638a-4581-9c84-5a2ea290ec81\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") "
Oct 30 19:39:15.989870 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:15.989850    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-credential-socket\") pod \"664f1334-638a-4581-9c84-5a2ea290ec81\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") "
Oct 30 19:39:15.989991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:15.989897    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/664f1334-638a-4581-9c84-5a2ea290ec81-istio-token\" (UniqueName: \"kubernetes.io/projected/664f1334-638a-4581-9c84-5a2ea290ec81-istio-token\") pod \"664f1334-638a-4581-9c84-5a2ea290ec81\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") "
Oct 30 19:39:15.989991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:15.989931    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-istio-data\") pod \"664f1334-638a-4581-9c84-5a2ea290ec81\" (UID: \"664f1334-638a-4581-9c84-5a2ea290ec81\") "
Oct 30 19:39:15.992178 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:15.992148    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/664f1334-638a-4581-9c84-5a2ea290ec81-istiod-ca-cert" pod "664f1334-638a-4581-9c84-5a2ea290ec81" (UID: "664f1334-638a-4581-9c84-5a2ea290ec81"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:39:15.993036 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:15.993011    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-workload-socket" pod "664f1334-638a-4581-9c84-5a2ea290ec81" (UID: "664f1334-638a-4581-9c84-5a2ea290ec81"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:15.994471 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:15.994199    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/664f1334-638a-4581-9c84-5a2ea290ec81-kube-api-access-85bsq" pod "664f1334-638a-4581-9c84-5a2ea290ec81" (UID: "664f1334-638a-4581-9c84-5a2ea290ec81"). InnerVolumeSpecName "kube-api-access-85bsq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:39:15.994840 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:15.994813    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-credential-socket" pod "664f1334-638a-4581-9c84-5a2ea290ec81" (UID: "664f1334-638a-4581-9c84-5a2ea290ec81"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:15.995043 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:15.995024    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-workload-certs" pod "664f1334-638a-4581-9c84-5a2ea290ec81" (UID: "664f1334-638a-4581-9c84-5a2ea290ec81"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:15.996864 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:15.996835    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-istio-data" pod "664f1334-638a-4581-9c84-5a2ea290ec81" (UID: "664f1334-638a-4581-9c84-5a2ea290ec81"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:16.029238 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.029160    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/664f1334-638a-4581-9c84-5a2ea290ec81-istio-podinfo" pod "664f1334-638a-4581-9c84-5a2ea290ec81" (UID: "664f1334-638a-4581-9c84-5a2ea290ec81"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:39:16.029766 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.029732    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-istio-envoy" pod "664f1334-638a-4581-9c84-5a2ea290ec81" (UID: "664f1334-638a-4581-9c84-5a2ea290ec81"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:16.030827 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.030374    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:16.036142 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.036113    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/664f1334-638a-4581-9c84-5a2ea290ec81-istio-token" pod "664f1334-638a-4581-9c84-5a2ea290ec81" (UID: "664f1334-638a-4581-9c84-5a2ea290ec81"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:39:16.091553 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.091485    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/664f1334-638a-4581-9c84-5a2ea290ec81-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:16.091553 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.091512    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/664f1334-638a-4581-9c84-5a2ea290ec81-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:16.091553 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.091522    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:16.091553 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.091532    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:16.091553 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.091540    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/664f1334-638a-4581-9c84-5a2ea290ec81-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:16.091553 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.091548    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:16.091876 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.091557    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85bsq\" (UniqueName: \"kubernetes.io/projected/664f1334-638a-4581-9c84-5a2ea290ec81-kube-api-access-85bsq\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:16.091876 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.091568    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:16.091876 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.091576    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/664f1334-638a-4581-9c84-5a2ea290ec81-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:16.342758 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.342670    5820 generic.go:356] "Generic (PLEG): container finished" podID="664f1334-638a-4581-9c84-5a2ea290ec81" containerID="5701229365ac15f91c11a99f8b93cb54085647dbb13efcf784fd73e507414ea0" exitCode=0
Oct 30 19:39:16.342758 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.342717    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"
Oct 30 19:39:16.342758 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.342748    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x" event={"ID":"664f1334-638a-4581-9c84-5a2ea290ec81","Type":"ContainerDied","Data":"5701229365ac15f91c11a99f8b93cb54085647dbb13efcf784fd73e507414ea0"}
Oct 30 19:39:16.343067 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.342806    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x" event={"ID":"664f1334-638a-4581-9c84-5a2ea290ec81","Type":"ContainerDied","Data":"b2c360a42a74b7711378191d1465589fddd32325a51663c940ff46af7cccb3c7"}
Oct 30 19:39:16.343067 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.342829    5820 scope.go:122] "RemoveContainer" containerID="5701229365ac15f91c11a99f8b93cb54085647dbb13efcf784fd73e507414ea0"
Oct 30 19:39:16.353376 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.353360    5820 scope.go:122] "RemoveContainer" containerID="2eb0cdd4b9da0e2f19c6ac3d6ea85af27aab70677f49711a538c0ad9b5f9c2b7"
Oct 30 19:39:16.362130 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.362115    5820 scope.go:122] "RemoveContainer" containerID="5701229365ac15f91c11a99f8b93cb54085647dbb13efcf784fd73e507414ea0"
Oct 30 19:39:16.362395 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:39:16.362379    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5701229365ac15f91c11a99f8b93cb54085647dbb13efcf784fd73e507414ea0\": container with ID starting with 5701229365ac15f91c11a99f8b93cb54085647dbb13efcf784fd73e507414ea0 not found: ID does not exist" containerID="5701229365ac15f91c11a99f8b93cb54085647dbb13efcf784fd73e507414ea0"
Oct 30 19:39:16.362437 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.362403    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5701229365ac15f91c11a99f8b93cb54085647dbb13efcf784fd73e507414ea0"} err="failed to get container status \"5701229365ac15f91c11a99f8b93cb54085647dbb13efcf784fd73e507414ea0\": rpc error: code = NotFound desc = could not find container \"5701229365ac15f91c11a99f8b93cb54085647dbb13efcf784fd73e507414ea0\": container with ID starting with 5701229365ac15f91c11a99f8b93cb54085647dbb13efcf784fd73e507414ea0 not found: ID does not exist"
Oct 30 19:39:16.362437 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.362421    5820 scope.go:122] "RemoveContainer" containerID="2eb0cdd4b9da0e2f19c6ac3d6ea85af27aab70677f49711a538c0ad9b5f9c2b7"
Oct 30 19:39:16.362667 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:39:16.362649    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eb0cdd4b9da0e2f19c6ac3d6ea85af27aab70677f49711a538c0ad9b5f9c2b7\": container with ID starting with 2eb0cdd4b9da0e2f19c6ac3d6ea85af27aab70677f49711a538c0ad9b5f9c2b7 not found: ID does not exist" containerID="2eb0cdd4b9da0e2f19c6ac3d6ea85af27aab70677f49711a538c0ad9b5f9c2b7"
Oct 30 19:39:16.362705 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.362673    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eb0cdd4b9da0e2f19c6ac3d6ea85af27aab70677f49711a538c0ad9b5f9c2b7"} err="failed to get container status \"2eb0cdd4b9da0e2f19c6ac3d6ea85af27aab70677f49711a538c0ad9b5f9c2b7\": rpc error: code = NotFound desc = could not find container \"2eb0cdd4b9da0e2f19c6ac3d6ea85af27aab70677f49711a538c0ad9b5f9c2b7\": container with ID starting with 2eb0cdd4b9da0e2f19c6ac3d6ea85af27aab70677f49711a538c0ad9b5f9c2b7 not found: ID does not exist"
Oct 30 19:39:16.510675 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.510646    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"]
Oct 30 19:39:16.514744 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:16.514711    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-cdd844-openshift-default-6dbcfb75cc-m5l8x"]
Oct 30 19:39:17.438154 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:17.438077    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="664f1334-638a-4581-9c84-5a2ea290ec81" path="/var/lib/kubelet/pods/664f1334-638a-4581-9c84-5a2ea290ec81/volumes"
Oct 30 19:39:18.016405 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:18.016352    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:18.462487 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:18.453015    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:19.430209 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:19.430162    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:19.666888 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:19.666848    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:20.575552 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:20.575521    5820 patch_prober.go:29] interesting pod/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:39:20.575724 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:20.575574    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x" podUID="c6c5724f-a1ce-448e-9535-3bbc49743b18" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:39:21.203512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.203479    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:39:21.250280 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.250248    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-data\") pod \"c6c5724f-a1ce-448e-9535-3bbc49743b18\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") "
Oct 30 19:39:21.250461 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.250305    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c6c5724f-a1ce-448e-9535-3bbc49743b18-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6c5724f-a1ce-448e-9535-3bbc49743b18-istiod-ca-cert\") pod \"c6c5724f-a1ce-448e-9535-3bbc49743b18\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") "
Oct 30 19:39:21.250461 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.250359    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-workload-certs\") pod \"c6c5724f-a1ce-448e-9535-3bbc49743b18\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") "
Oct 30 19:39:21.250461 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.250399    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-podinfo\") pod \"c6c5724f-a1ce-448e-9535-3bbc49743b18\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") "
Oct 30 19:39:21.250614 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.250476    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-credential-socket\") pod \"c6c5724f-a1ce-448e-9535-3bbc49743b18\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") "
Oct 30 19:39:21.250614 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.250515    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-envoy\") pod \"c6c5724f-a1ce-448e-9535-3bbc49743b18\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") "
Oct 30 19:39:21.250614 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.250534    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-data" pod "c6c5724f-a1ce-448e-9535-3bbc49743b18" (UID: "c6c5724f-a1ce-448e-9535-3bbc49743b18"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:21.250614 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.250554    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-token\" (UniqueName: \"kubernetes.io/projected/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-token\") pod \"c6c5724f-a1ce-448e-9535-3bbc49743b18\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") "
Oct 30 19:39:21.250614 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.250610    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6c5724f-a1ce-448e-9535-3bbc49743b18-kube-api-access-mdll2\" (UniqueName: \"kubernetes.io/projected/c6c5724f-a1ce-448e-9535-3bbc49743b18-kube-api-access-mdll2\") pod \"c6c5724f-a1ce-448e-9535-3bbc49743b18\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") "
Oct 30 19:39:21.250889 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.250673    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-workload-socket\") pod \"c6c5724f-a1ce-448e-9535-3bbc49743b18\" (UID: \"c6c5724f-a1ce-448e-9535-3bbc49743b18\") "
Oct 30 19:39:21.250889 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.250718    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-workload-certs" pod "c6c5724f-a1ce-448e-9535-3bbc49743b18" (UID: "c6c5724f-a1ce-448e-9535-3bbc49743b18"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:21.250889 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.250771    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6c5724f-a1ce-448e-9535-3bbc49743b18-istiod-ca-cert" pod "c6c5724f-a1ce-448e-9535-3bbc49743b18" (UID: "c6c5724f-a1ce-448e-9535-3bbc49743b18"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:39:21.251093 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.251006    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-workload-socket" pod "c6c5724f-a1ce-448e-9535-3bbc49743b18" (UID: "c6c5724f-a1ce-448e-9535-3bbc49743b18"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:21.251154 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.251100    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-credential-socket" pod "c6c5724f-a1ce-448e-9535-3bbc49743b18" (UID: "c6c5724f-a1ce-448e-9535-3bbc49743b18"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:21.251231 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.251206    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:21.251231 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.251220    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:21.251311 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.251229    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:21.251311 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.251241    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6c5724f-a1ce-448e-9535-3bbc49743b18-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:21.251311 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.251251    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:21.252905 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.252875    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-envoy" pod "c6c5724f-a1ce-448e-9535-3bbc49743b18" (UID: "c6c5724f-a1ce-448e-9535-3bbc49743b18"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:21.253077 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.252929    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-podinfo" pod "c6c5724f-a1ce-448e-9535-3bbc49743b18" (UID: "c6c5724f-a1ce-448e-9535-3bbc49743b18"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:39:21.253135 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.253108    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6c5724f-a1ce-448e-9535-3bbc49743b18-kube-api-access-mdll2" pod "c6c5724f-a1ce-448e-9535-3bbc49743b18" (UID: "c6c5724f-a1ce-448e-9535-3bbc49743b18"). InnerVolumeSpecName "kube-api-access-mdll2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:39:21.253253 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.253236    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-token" pod "c6c5724f-a1ce-448e-9535-3bbc49743b18" (UID: "c6c5724f-a1ce-448e-9535-3bbc49743b18"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:39:21.352104 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.352073    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:21.352104 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.352101    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:21.352345 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.352114    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdll2\" (UniqueName: \"kubernetes.io/projected/c6c5724f-a1ce-448e-9535-3bbc49743b18-kube-api-access-mdll2\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:21.352345 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.352128    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6c5724f-a1ce-448e-9535-3bbc49743b18-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:21.389840 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.389802    5820 generic.go:356] "Generic (PLEG): container finished" podID="c6c5724f-a1ce-448e-9535-3bbc49743b18" containerID="6cf5ae072abdb13e2fd89a61d63c0da21a75befb45b000deea95bcebf0e6e001" exitCode=0
Oct 30 19:39:21.390067 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.389880    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"
Oct 30 19:39:21.390141 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.390063    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x" event={"ID":"c6c5724f-a1ce-448e-9535-3bbc49743b18","Type":"ContainerDied","Data":"6cf5ae072abdb13e2fd89a61d63c0da21a75befb45b000deea95bcebf0e6e001"}
Oct 30 19:39:21.390214 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.390147    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x" event={"ID":"c6c5724f-a1ce-448e-9535-3bbc49743b18","Type":"ContainerDied","Data":"1f7b0c1dcb40fa23a02807ec03eac0a77fe8b988087bd4ee604ad727905918df"}
Oct 30 19:39:21.390214 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.390180    5820 scope.go:122] "RemoveContainer" containerID="6cf5ae072abdb13e2fd89a61d63c0da21a75befb45b000deea95bcebf0e6e001"
Oct 30 19:39:21.405399 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.405372    5820 scope.go:122] "RemoveContainer" containerID="2fe5dcdb9e6357bd512e6ff42239be39f43fe0c0a50110c34a622d418eff7458"
Oct 30 19:39:21.415568 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.415545    5820 scope.go:122] "RemoveContainer" containerID="6cf5ae072abdb13e2fd89a61d63c0da21a75befb45b000deea95bcebf0e6e001"
Oct 30 19:39:21.415893 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:39:21.415868    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cf5ae072abdb13e2fd89a61d63c0da21a75befb45b000deea95bcebf0e6e001\": container with ID starting with 6cf5ae072abdb13e2fd89a61d63c0da21a75befb45b000deea95bcebf0e6e001 not found: ID does not exist" containerID="6cf5ae072abdb13e2fd89a61d63c0da21a75befb45b000deea95bcebf0e6e001"
Oct 30 19:39:21.416001 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.415905    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cf5ae072abdb13e2fd89a61d63c0da21a75befb45b000deea95bcebf0e6e001"} err="failed to get container status \"6cf5ae072abdb13e2fd89a61d63c0da21a75befb45b000deea95bcebf0e6e001\": rpc error: code = NotFound desc = could not find container \"6cf5ae072abdb13e2fd89a61d63c0da21a75befb45b000deea95bcebf0e6e001\": container with ID starting with 6cf5ae072abdb13e2fd89a61d63c0da21a75befb45b000deea95bcebf0e6e001 not found: ID does not exist"
Oct 30 19:39:21.416001 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.415924    5820 scope.go:122] "RemoveContainer" containerID="2fe5dcdb9e6357bd512e6ff42239be39f43fe0c0a50110c34a622d418eff7458"
Oct 30 19:39:21.416200 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:39:21.416183    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fe5dcdb9e6357bd512e6ff42239be39f43fe0c0a50110c34a622d418eff7458\": container with ID starting with 2fe5dcdb9e6357bd512e6ff42239be39f43fe0c0a50110c34a622d418eff7458 not found: ID does not exist" containerID="2fe5dcdb9e6357bd512e6ff42239be39f43fe0c0a50110c34a622d418eff7458"
Oct 30 19:39:21.416247 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.416205    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fe5dcdb9e6357bd512e6ff42239be39f43fe0c0a50110c34a622d418eff7458"} err="failed to get container status \"2fe5dcdb9e6357bd512e6ff42239be39f43fe0c0a50110c34a622d418eff7458\": rpc error: code = NotFound desc = could not find container \"2fe5dcdb9e6357bd512e6ff42239be39f43fe0c0a50110c34a622d418eff7458\": container with ID starting with 2fe5dcdb9e6357bd512e6ff42239be39f43fe0c0a50110c34a622d418eff7458 not found: ID does not exist"
Oct 30 19:39:21.556096 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.556058    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"]
Oct 30 19:39:21.563474 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:21.563445    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-0f83f5-openshift-default-657d4bc56f-fhr6x"]
Oct 30 19:39:22.720315 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:22.720283    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:23.380414 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:23.380375    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:23.436910 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:23.436879    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c6c5724f-a1ce-448e-9535-3bbc49743b18" path="/var/lib/kubelet/pods/c6c5724f-a1ce-448e-9535-3bbc49743b18/volumes"
Oct 30 19:39:23.606547 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:23.606509    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:23.944711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:23.944678    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:24.166814 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:24.166773    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:24.357265 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:24.357214    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:24.674343 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:24.674258    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:24.910691 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:24.910662    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:25.217182 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:25.217147    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:25.777939 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:25.777888    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:27.517794 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:27.517759    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:27.749988 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:27.749928    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:30.019904 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:30.019864    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:32.712126 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:32.712094    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:32.906345 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:32.906304    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:33.147519 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:33.147481    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:35.385048 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:35.385009    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:38.203791 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:39:38.203752    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:39:40.649313 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:40.649169    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:45.521230 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:45.521195    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd"]
Oct 30 19:39:45.523804 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:45.523771    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:45.630116 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:45.630079    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74"]
Oct 30 19:39:45.961379 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:45.961280    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:46.182826 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:46.182791    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:50.340438 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:50.340324    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:50.522399 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:50.522339    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd" podUID="349f7a4a-cae3-41b7-a6dd-ead82d9b0a64" containerName="payload-processing" containerID="cri-o://99f87264fe6edfd2398466ae9afa612a733bfcd7c1b5d80b99eac2d288db4c27" gracePeriod=25
Oct 30 19:39:50.630990 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:50.630883    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74" podUID="336a57c2-7270-4cd3-843f-b8a63a26bcab" containerName="payload-pre-processing" containerID="cri-o://02be29a76ccecb6927fb8f06b7a5c8407e9f75500d93041668d5714dcce42387" gracePeriod=25
Oct 30 19:39:55.226651 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:55.225036    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:56.735121 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:56.735080    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd" podUID="349f7a4a-cae3-41b7-a6dd-ead82d9b0a64" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:39:56.735509 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:56.735135    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74" podUID="336a57c2-7270-4cd3-843f-b8a63a26bcab" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:39:59.382691 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:39:59.382645    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:00.779965 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:00.779909    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:01.012028 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:01.011989    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:01.196976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:01.196925    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:01.393994 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:01.393924    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:02.035710 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:02.035676    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:06.389429 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:06.389391    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:06.734328 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:06.734246    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd" podUID="349f7a4a-cae3-41b7-a6dd-ead82d9b0a64" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:40:06.734496 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:06.734249    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74" podUID="336a57c2-7270-4cd3-843f-b8a63a26bcab" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:40:08.203745 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:40:08.203703    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:40:08.428068 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:08.428032    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:08.659442 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:08.659401    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:08.861113 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:08.861079    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:09.046752 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:09.046659    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:10.717485 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:10.717447    5820 generic.go:356] "Generic (PLEG): container finished" podID="349f7a4a-cae3-41b7-a6dd-ead82d9b0a64" containerID="99f87264fe6edfd2398466ae9afa612a733bfcd7c1b5d80b99eac2d288db4c27" exitCode=0
Oct 30 19:40:10.717899 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:10.717524    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd" event={"ID":"349f7a4a-cae3-41b7-a6dd-ead82d9b0a64","Type":"ContainerDied","Data":"99f87264fe6edfd2398466ae9afa612a733bfcd7c1b5d80b99eac2d288db4c27"}
Oct 30 19:40:10.722856 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:10.722821    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:10.817237 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:10.816974    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd"
Oct 30 19:40:10.911529 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:10.911504    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74"
Oct 30 19:40:10.964763 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:10.964730    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/349f7a4a-cae3-41b7-a6dd-ead82d9b0a64-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/349f7a4a-cae3-41b7-a6dd-ead82d9b0a64-plugins-config-volume\") pod \"349f7a4a-cae3-41b7-a6dd-ead82d9b0a64\" (UID: \"349f7a4a-cae3-41b7-a6dd-ead82d9b0a64\") "
Oct 30 19:40:10.965125 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:10.965101    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/349f7a4a-cae3-41b7-a6dd-ead82d9b0a64-kube-api-access-2dglj\" (UniqueName: \"kubernetes.io/projected/349f7a4a-cae3-41b7-a6dd-ead82d9b0a64-kube-api-access-2dglj\") pod \"349f7a4a-cae3-41b7-a6dd-ead82d9b0a64\" (UID: \"349f7a4a-cae3-41b7-a6dd-ead82d9b0a64\") "
Oct 30 19:40:10.965460 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:10.965383    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/349f7a4a-cae3-41b7-a6dd-ead82d9b0a64-plugins-config-volume" pod "349f7a4a-cae3-41b7-a6dd-ead82d9b0a64" (UID: "349f7a4a-cae3-41b7-a6dd-ead82d9b0a64"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:40:10.965909 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:10.965882    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/349f7a4a-cae3-41b7-a6dd-ead82d9b0a64-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:10.968579 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:10.968555    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/349f7a4a-cae3-41b7-a6dd-ead82d9b0a64-kube-api-access-2dglj" pod "349f7a4a-cae3-41b7-a6dd-ead82d9b0a64" (UID: "349f7a4a-cae3-41b7-a6dd-ead82d9b0a64"). InnerVolumeSpecName "kube-api-access-2dglj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:11.066884 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:11.066856    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/336a57c2-7270-4cd3-843f-b8a63a26bcab-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/336a57c2-7270-4cd3-843f-b8a63a26bcab-plugins-config-volume\") pod \"336a57c2-7270-4cd3-843f-b8a63a26bcab\" (UID: \"336a57c2-7270-4cd3-843f-b8a63a26bcab\") "
Oct 30 19:40:11.067083 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:11.067010    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/336a57c2-7270-4cd3-843f-b8a63a26bcab-kube-api-access-mfmdz\" (UniqueName: \"kubernetes.io/projected/336a57c2-7270-4cd3-843f-b8a63a26bcab-kube-api-access-mfmdz\") pod \"336a57c2-7270-4cd3-843f-b8a63a26bcab\" (UID: \"336a57c2-7270-4cd3-843f-b8a63a26bcab\") "
Oct 30 19:40:11.067363 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:11.067308    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/336a57c2-7270-4cd3-843f-b8a63a26bcab-plugins-config-volume" pod "336a57c2-7270-4cd3-843f-b8a63a26bcab" (UID: "336a57c2-7270-4cd3-843f-b8a63a26bcab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:40:11.067456 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:11.067429    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dglj\" (UniqueName: \"kubernetes.io/projected/349f7a4a-cae3-41b7-a6dd-ead82d9b0a64-kube-api-access-2dglj\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:11.067456 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:11.067449    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/336a57c2-7270-4cd3-843f-b8a63a26bcab-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:11.082282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:11.081092    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/336a57c2-7270-4cd3-843f-b8a63a26bcab-kube-api-access-mfmdz" pod "336a57c2-7270-4cd3-843f-b8a63a26bcab" (UID: "336a57c2-7270-4cd3-843f-b8a63a26bcab"). InnerVolumeSpecName "kube-api-access-mfmdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:11.169095 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:11.169058    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfmdz\" (UniqueName: \"kubernetes.io/projected/336a57c2-7270-4cd3-843f-b8a63a26bcab-kube-api-access-mfmdz\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:11.725193 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:11.725112    5820 generic.go:356] "Generic (PLEG): container finished" podID="336a57c2-7270-4cd3-843f-b8a63a26bcab" containerID="02be29a76ccecb6927fb8f06b7a5c8407e9f75500d93041668d5714dcce42387" exitCode=0
Oct 30 19:40:11.725193 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:11.725159    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74"
Oct 30 19:40:11.725724 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:11.725193    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74" event={"ID":"336a57c2-7270-4cd3-843f-b8a63a26bcab","Type":"ContainerDied","Data":"02be29a76ccecb6927fb8f06b7a5c8407e9f75500d93041668d5714dcce42387"}
Oct 30 19:40:11.725724 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:11.725257    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74" event={"ID":"336a57c2-7270-4cd3-843f-b8a63a26bcab","Type":"ContainerDied","Data":"951742578cf402ac5b0684c959ebd3e2e14abefc374022b4c9b735eafa8d5d4d"}
Oct 30 19:40:11.725724 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:11.725291    5820 scope.go:122] "RemoveContainer" containerID="02be29a76ccecb6927fb8f06b7a5c8407e9f75500d93041668d5714dcce42387"
Oct 30 19:40:11.726777 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:11.726756    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd" event={"ID":"349f7a4a-cae3-41b7-a6dd-ead82d9b0a64","Type":"ContainerDied","Data":"e28321a205e48d0d139305d27824d5207d4086a207ea61f0633c097f7fabaf88"}
Oct 30 19:40:11.726864 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:11.726764    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd"
Oct 30 19:40:11.735271 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:11.735253    5820 scope.go:122] "RemoveContainer" containerID="02be29a76ccecb6927fb8f06b7a5c8407e9f75500d93041668d5714dcce42387"
Oct 30 19:40:11.735553 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:40:11.735531    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02be29a76ccecb6927fb8f06b7a5c8407e9f75500d93041668d5714dcce42387\": container with ID starting with 02be29a76ccecb6927fb8f06b7a5c8407e9f75500d93041668d5714dcce42387 not found: ID does not exist" containerID="02be29a76ccecb6927fb8f06b7a5c8407e9f75500d93041668d5714dcce42387"
Oct 30 19:40:11.735622 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:11.735562    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02be29a76ccecb6927fb8f06b7a5c8407e9f75500d93041668d5714dcce42387"} err="failed to get container status \"02be29a76ccecb6927fb8f06b7a5c8407e9f75500d93041668d5714dcce42387\": rpc error: code = NotFound desc = could not find container \"02be29a76ccecb6927fb8f06b7a5c8407e9f75500d93041668d5714dcce42387\": container with ID starting with 02be29a76ccecb6927fb8f06b7a5c8407e9f75500d93041668d5714dcce42387 not found: ID does not exist"
Oct 30 19:40:11.735622 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:11.735578    5820 scope.go:122] "RemoveContainer" containerID="99f87264fe6edfd2398466ae9afa612a733bfcd7c1b5d80b99eac2d288db4c27"
Oct 30 19:40:11.890536 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:11.890507    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74"]
Oct 30 19:40:11.892984 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:11.892936    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8f9c6b-6479f9dd79-qpx74"]
Oct 30 19:40:12.041892 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:12.041823    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd"]
Oct 30 19:40:12.077555 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:12.077519    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8f9c6b-5fc89d55dd-rdbmd"]
Oct 30 19:40:12.336466 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:12.336432    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:12.581003 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:12.580928    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:12.764113 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:12.764027    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:13.436906 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:13.436874    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="336a57c2-7270-4cd3-843f-b8a63a26bcab" path="/var/lib/kubelet/pods/336a57c2-7270-4cd3-843f-b8a63a26bcab/volumes"
Oct 30 19:40:13.437277 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:13.437264    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="349f7a4a-cae3-41b7-a6dd-ead82d9b0a64" path="/var/lib/kubelet/pods/349f7a4a-cae3-41b7-a6dd-ead82d9b0a64/volumes"
Oct 30 19:40:13.480404 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:13.480367    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:16.922841 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:16.922811    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:17.027219 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.027187    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"]
Oct 30 19:40:17.027449 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.027424    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv" podUID="a5f8e8dd-a036-477e-ac6e-b2f15500af33" containerName="maas-api" containerID="cri-o://91fe75abd28985b7d8afe674574b16edb5589f9854d30b353100b7c74bd4d913" gracePeriod=30
Oct 30 19:40:17.184078 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.183994    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:17.284720 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.284698    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"
Oct 30 19:40:17.344547 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.344514    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a5f8e8dd-a036-477e-ac6e-b2f15500af33-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5f8e8dd-a036-477e-ac6e-b2f15500af33-metrics-certs\") pod \"a5f8e8dd-a036-477e-ac6e-b2f15500af33\" (UID: \"a5f8e8dd-a036-477e-ac6e-b2f15500af33\") "
Oct 30 19:40:17.344724 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.344583    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a5f8e8dd-a036-477e-ac6e-b2f15500af33-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5f8e8dd-a036-477e-ac6e-b2f15500af33-maas-api-tls\") pod \"a5f8e8dd-a036-477e-ac6e-b2f15500af33\" (UID: \"a5f8e8dd-a036-477e-ac6e-b2f15500af33\") "
Oct 30 19:40:17.344724 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.344704    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5f8e8dd-a036-477e-ac6e-b2f15500af33-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5f8e8dd-a036-477e-ac6e-b2f15500af33-openshift-service-ca\") pod \"a5f8e8dd-a036-477e-ac6e-b2f15500af33\" (UID: \"a5f8e8dd-a036-477e-ac6e-b2f15500af33\") "
Oct 30 19:40:17.344808 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.344751    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5f8e8dd-a036-477e-ac6e-b2f15500af33-kube-api-access-pwc49\" (UniqueName: \"kubernetes.io/projected/a5f8e8dd-a036-477e-ac6e-b2f15500af33-kube-api-access-pwc49\") pod \"a5f8e8dd-a036-477e-ac6e-b2f15500af33\" (UID: \"a5f8e8dd-a036-477e-ac6e-b2f15500af33\") "
Oct 30 19:40:17.345138 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.345111    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5f8e8dd-a036-477e-ac6e-b2f15500af33-openshift-service-ca" pod "a5f8e8dd-a036-477e-ac6e-b2f15500af33" (UID: "a5f8e8dd-a036-477e-ac6e-b2f15500af33"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:40:17.345431 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.345241    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5f8e8dd-a036-477e-ac6e-b2f15500af33-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:17.346846 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.346826    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5f8e8dd-a036-477e-ac6e-b2f15500af33-metrics-certs" pod "a5f8e8dd-a036-477e-ac6e-b2f15500af33" (UID: "a5f8e8dd-a036-477e-ac6e-b2f15500af33"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:40:17.347035 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.346879    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5f8e8dd-a036-477e-ac6e-b2f15500af33-maas-api-tls" pod "a5f8e8dd-a036-477e-ac6e-b2f15500af33" (UID: "a5f8e8dd-a036-477e-ac6e-b2f15500af33"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:40:17.347035 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.347009    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5f8e8dd-a036-477e-ac6e-b2f15500af33-kube-api-access-pwc49" pod "a5f8e8dd-a036-477e-ac6e-b2f15500af33" (UID: "a5f8e8dd-a036-477e-ac6e-b2f15500af33"). InnerVolumeSpecName "kube-api-access-pwc49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:17.446025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.445997    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwc49\" (UniqueName: \"kubernetes.io/projected/a5f8e8dd-a036-477e-ac6e-b2f15500af33-kube-api-access-pwc49\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:17.446025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.446022    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5f8e8dd-a036-477e-ac6e-b2f15500af33-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:17.446243 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.446032    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5f8e8dd-a036-477e-ac6e-b2f15500af33-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:17.769718 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.769616    5820 generic.go:356] "Generic (PLEG): container finished" podID="a5f8e8dd-a036-477e-ac6e-b2f15500af33" containerID="91fe75abd28985b7d8afe674574b16edb5589f9854d30b353100b7c74bd4d913" exitCode=0
Oct 30 19:40:17.769718 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.769683    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"
Oct 30 19:40:17.770048 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.769700    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv" event={"ID":"a5f8e8dd-a036-477e-ac6e-b2f15500af33","Type":"ContainerDied","Data":"91fe75abd28985b7d8afe674574b16edb5589f9854d30b353100b7c74bd4d913"}
Oct 30 19:40:17.770048 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.769786    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv" event={"ID":"a5f8e8dd-a036-477e-ac6e-b2f15500af33","Type":"ContainerDied","Data":"83fc05809407852b83a4b236756f9affdae30289f9fe7e8154131940cfcac60f"}
Oct 30 19:40:17.770048 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.769827    5820 scope.go:122] "RemoveContainer" containerID="91fe75abd28985b7d8afe674574b16edb5589f9854d30b353100b7c74bd4d913"
Oct 30 19:40:17.781929 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.781905    5820 scope.go:122] "RemoveContainer" containerID="91fe75abd28985b7d8afe674574b16edb5589f9854d30b353100b7c74bd4d913"
Oct 30 19:40:17.782263 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:40:17.782237    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91fe75abd28985b7d8afe674574b16edb5589f9854d30b353100b7c74bd4d913\": container with ID starting with 91fe75abd28985b7d8afe674574b16edb5589f9854d30b353100b7c74bd4d913 not found: ID does not exist" containerID="91fe75abd28985b7d8afe674574b16edb5589f9854d30b353100b7c74bd4d913"
Oct 30 19:40:17.782329 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.782275    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91fe75abd28985b7d8afe674574b16edb5589f9854d30b353100b7c74bd4d913"} err="failed to get container status \"91fe75abd28985b7d8afe674574b16edb5589f9854d30b353100b7c74bd4d913\": rpc error: code = NotFound desc = could not find container \"91fe75abd28985b7d8afe674574b16edb5589f9854d30b353100b7c74bd4d913\": container with ID starting with 91fe75abd28985b7d8afe674574b16edb5589f9854d30b353100b7c74bd4d913 not found: ID does not exist"
Oct 30 19:40:17.934645 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.934614    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"]
Oct 30 19:40:17.938496 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:17.938465    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-25e5ef-795cb55647-p96pv"]
Oct 30 19:40:18.797325 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:18.797286    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:19.027826 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:19.027791    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:19.203262 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:19.203227    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:19.437169 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:19.437136    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5f8e8dd-a036-477e-ac6e-b2f15500af33" path="/var/lib/kubelet/pods/a5f8e8dd-a036-477e-ac6e-b2f15500af33/volumes"
Oct 30 19:40:19.727716 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:19.727674    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:21.024711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:21.024670    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:21.275702 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:21.275611    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:21.465906 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:21.465868    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:21.659277 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:21.659225    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:21.887925 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:21.887878    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:24.306542 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:24.306500    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:26.227745 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:26.227703    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:26.404710 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:26.404674    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:26.575331 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:26.575295    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:29.571522 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:29.571482    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:29.803723 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:29.803690    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:29.995172 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:29.995092    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:34.268218 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.268182    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"]
Oct 30 19:40:34.268650 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.268387    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp" podUID="612e60bf-47da-407a-b30d-6213966bd7a9" containerName="maas-api" containerID="cri-o://fb9539f54fdf77b6412d22fb1e11e2f443dd93cdb749f0776df49ed7b77f18b6" gracePeriod=30
Oct 30 19:40:34.409812 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.409774    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:34.545414 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.545389    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"
Oct 30 19:40:34.652536 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.652494    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/612e60bf-47da-407a-b30d-6213966bd7a9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/612e60bf-47da-407a-b30d-6213966bd7a9-metrics-certs\") pod \"612e60bf-47da-407a-b30d-6213966bd7a9\" (UID: \"612e60bf-47da-407a-b30d-6213966bd7a9\") "
Oct 30 19:40:34.652760 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.652575    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/612e60bf-47da-407a-b30d-6213966bd7a9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/612e60bf-47da-407a-b30d-6213966bd7a9-maas-api-tls\") pod \"612e60bf-47da-407a-b30d-6213966bd7a9\" (UID: \"612e60bf-47da-407a-b30d-6213966bd7a9\") "
Oct 30 19:40:34.652760 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.652622    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/612e60bf-47da-407a-b30d-6213966bd7a9-kube-api-access-spk2n\" (UniqueName: \"kubernetes.io/projected/612e60bf-47da-407a-b30d-6213966bd7a9-kube-api-access-spk2n\") pod \"612e60bf-47da-407a-b30d-6213966bd7a9\" (UID: \"612e60bf-47da-407a-b30d-6213966bd7a9\") "
Oct 30 19:40:34.652760 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.652705    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/612e60bf-47da-407a-b30d-6213966bd7a9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/612e60bf-47da-407a-b30d-6213966bd7a9-openshift-service-ca\") pod \"612e60bf-47da-407a-b30d-6213966bd7a9\" (UID: \"612e60bf-47da-407a-b30d-6213966bd7a9\") "
Oct 30 19:40:34.653587 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.653543    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/612e60bf-47da-407a-b30d-6213966bd7a9-openshift-service-ca" pod "612e60bf-47da-407a-b30d-6213966bd7a9" (UID: "612e60bf-47da-407a-b30d-6213966bd7a9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:40:34.656382 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.656345    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/612e60bf-47da-407a-b30d-6213966bd7a9-kube-api-access-spk2n" pod "612e60bf-47da-407a-b30d-6213966bd7a9" (UID: "612e60bf-47da-407a-b30d-6213966bd7a9"). InnerVolumeSpecName "kube-api-access-spk2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:34.656509 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.656398    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/612e60bf-47da-407a-b30d-6213966bd7a9-metrics-certs" pod "612e60bf-47da-407a-b30d-6213966bd7a9" (UID: "612e60bf-47da-407a-b30d-6213966bd7a9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:40:34.656509 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.656437    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/612e60bf-47da-407a-b30d-6213966bd7a9-maas-api-tls" pod "612e60bf-47da-407a-b30d-6213966bd7a9" (UID: "612e60bf-47da-407a-b30d-6213966bd7a9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:40:34.754289 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.754249    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/612e60bf-47da-407a-b30d-6213966bd7a9-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:34.754289 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.754287    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/612e60bf-47da-407a-b30d-6213966bd7a9-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:34.754531 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.754305    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spk2n\" (UniqueName: \"kubernetes.io/projected/612e60bf-47da-407a-b30d-6213966bd7a9-kube-api-access-spk2n\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:34.754531 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.754320    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/612e60bf-47da-407a-b30d-6213966bd7a9-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:34.879479 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.879437    5820 generic.go:356] "Generic (PLEG): container finished" podID="612e60bf-47da-407a-b30d-6213966bd7a9" containerID="fb9539f54fdf77b6412d22fb1e11e2f443dd93cdb749f0776df49ed7b77f18b6" exitCode=0
Oct 30 19:40:34.879680 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.879481    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"
Oct 30 19:40:34.879680 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.879499    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp" event={"ID":"612e60bf-47da-407a-b30d-6213966bd7a9","Type":"ContainerDied","Data":"fb9539f54fdf77b6412d22fb1e11e2f443dd93cdb749f0776df49ed7b77f18b6"}
Oct 30 19:40:34.879680 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.879552    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp" event={"ID":"612e60bf-47da-407a-b30d-6213966bd7a9","Type":"ContainerDied","Data":"d47cf82a10489c3b8ea920a3e86c86d3bdb415351e52cab809fbbc65f5db3568"}
Oct 30 19:40:34.879680 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.879582    5820 scope.go:122] "RemoveContainer" containerID="fb9539f54fdf77b6412d22fb1e11e2f443dd93cdb749f0776df49ed7b77f18b6"
Oct 30 19:40:34.890440 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.890421    5820 scope.go:122] "RemoveContainer" containerID="fb9539f54fdf77b6412d22fb1e11e2f443dd93cdb749f0776df49ed7b77f18b6"
Oct 30 19:40:34.890711 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:40:34.890692    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb9539f54fdf77b6412d22fb1e11e2f443dd93cdb749f0776df49ed7b77f18b6\": container with ID starting with fb9539f54fdf77b6412d22fb1e11e2f443dd93cdb749f0776df49ed7b77f18b6 not found: ID does not exist" containerID="fb9539f54fdf77b6412d22fb1e11e2f443dd93cdb749f0776df49ed7b77f18b6"
Oct 30 19:40:34.890760 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:34.890721    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb9539f54fdf77b6412d22fb1e11e2f443dd93cdb749f0776df49ed7b77f18b6"} err="failed to get container status \"fb9539f54fdf77b6412d22fb1e11e2f443dd93cdb749f0776df49ed7b77f18b6\": rpc error: code = NotFound desc = could not find container \"fb9539f54fdf77b6412d22fb1e11e2f443dd93cdb749f0776df49ed7b77f18b6\": container with ID starting with fb9539f54fdf77b6412d22fb1e11e2f443dd93cdb749f0776df49ed7b77f18b6 not found: ID does not exist"
Oct 30 19:40:35.045225 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:35.045199    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"]
Oct 30 19:40:35.048693 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:35.048661    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8f9c6b-7557cb5956-j85sp"]
Oct 30 19:40:35.436890 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:35.436860    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="612e60bf-47da-407a-b30d-6213966bd7a9" path="/var/lib/kubelet/pods/612e60bf-47da-407a-b30d-6213966bd7a9/volumes"
Oct 30 19:40:36.012480 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:36.012444    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:36.290418 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:36.290338    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:36.472686 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:36.472652    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:36.986437 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:36.986398    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:38.203036 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:40:38.203000    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:40:39.034597 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:39.034561    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw"]
Oct 30 19:40:39.328866 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:39.328828    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:39.505391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:39.505343    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:41.346763 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:41.346723    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:43.940906 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:43.940855    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:44.036211 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:44.036175    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw" podUID="bf3c4c7a-8bb7-440f-8d63-1d961dca8371" containerName="payload-pre-processing" containerID="cri-o://e7b981d4fc6545892f11d5ea9d889e78e423b5e5c11e9e722f232a2e55768cd0" gracePeriod=25
Oct 30 19:40:44.120525 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:44.120489    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:44.254546 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:44.254467    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"]
Oct 30 19:40:44.254758 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:44.254731    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2" podUID="795704e1-5f91-4636-a0b5-d9271dc4551d" containerName="istio-proxy" containerID="cri-o://3369c4799f7787fdff17f62a41cb6adfc659cc0180dde0c1278fd875ee84c46b" gracePeriod=30
Oct 30 19:40:44.315436 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:44.315392    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:46.314685 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:46.314645    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:49.513611 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.513584    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:40:49.621493 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.621462    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-credential-socket\") pod \"795704e1-5f91-4636-a0b5-d9271dc4551d\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") "
Oct 30 19:40:49.621657 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.621509    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/795704e1-5f91-4636-a0b5-d9271dc4551d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/795704e1-5f91-4636-a0b5-d9271dc4551d-istiod-ca-cert\") pod \"795704e1-5f91-4636-a0b5-d9271dc4551d\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") "
Oct 30 19:40:49.621657 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.621540    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-token\" (UniqueName: \"kubernetes.io/projected/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-token\") pod \"795704e1-5f91-4636-a0b5-d9271dc4551d\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") "
Oct 30 19:40:49.621788 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.621684    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-podinfo\") pod \"795704e1-5f91-4636-a0b5-d9271dc4551d\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") "
Oct 30 19:40:49.621788 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.621720    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-credential-socket" pod "795704e1-5f91-4636-a0b5-d9271dc4551d" (UID: "795704e1-5f91-4636-a0b5-d9271dc4551d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:49.621917 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.621827    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-workload-certs\") pod \"795704e1-5f91-4636-a0b5-d9271dc4551d\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") "
Oct 30 19:40:49.621917 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.621901    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-workload-socket\") pod \"795704e1-5f91-4636-a0b5-d9271dc4551d\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") "
Oct 30 19:40:49.622068 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.621906    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/795704e1-5f91-4636-a0b5-d9271dc4551d-istiod-ca-cert" pod "795704e1-5f91-4636-a0b5-d9271dc4551d" (UID: "795704e1-5f91-4636-a0b5-d9271dc4551d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:40:49.622068 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.621982    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-envoy\") pod \"795704e1-5f91-4636-a0b5-d9271dc4551d\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") "
Oct 30 19:40:49.622068 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.622015    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-data\") pod \"795704e1-5f91-4636-a0b5-d9271dc4551d\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") "
Oct 30 19:40:49.622068 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.622044    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/795704e1-5f91-4636-a0b5-d9271dc4551d-kube-api-access-5khrg\" (UniqueName: \"kubernetes.io/projected/795704e1-5f91-4636-a0b5-d9271dc4551d-kube-api-access-5khrg\") pod \"795704e1-5f91-4636-a0b5-d9271dc4551d\" (UID: \"795704e1-5f91-4636-a0b5-d9271dc4551d\") "
Oct 30 19:40:49.622290 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.622080    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-workload-certs" pod "795704e1-5f91-4636-a0b5-d9271dc4551d" (UID: "795704e1-5f91-4636-a0b5-d9271dc4551d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:49.622290 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.622227    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-workload-socket" pod "795704e1-5f91-4636-a0b5-d9271dc4551d" (UID: "795704e1-5f91-4636-a0b5-d9271dc4551d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:49.622784 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.622635    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:49.622784 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.622662    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:49.622784 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.622716    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/795704e1-5f91-4636-a0b5-d9271dc4551d-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:49.623153 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.622853    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:49.623153 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.622998    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-data" pod "795704e1-5f91-4636-a0b5-d9271dc4551d" (UID: "795704e1-5f91-4636-a0b5-d9271dc4551d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:49.624259 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.624229    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-token" pod "795704e1-5f91-4636-a0b5-d9271dc4551d" (UID: "795704e1-5f91-4636-a0b5-d9271dc4551d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:49.624387 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.624269    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-podinfo" pod "795704e1-5f91-4636-a0b5-d9271dc4551d" (UID: "795704e1-5f91-4636-a0b5-d9271dc4551d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:40:49.624464 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.624387    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-envoy" pod "795704e1-5f91-4636-a0b5-d9271dc4551d" (UID: "795704e1-5f91-4636-a0b5-d9271dc4551d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:49.624715 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.624699    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/795704e1-5f91-4636-a0b5-d9271dc4551d-kube-api-access-5khrg" pod "795704e1-5f91-4636-a0b5-d9271dc4551d" (UID: "795704e1-5f91-4636-a0b5-d9271dc4551d"). InnerVolumeSpecName "kube-api-access-5khrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:49.724096 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.724063    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:49.724096 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.724089    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:49.724096 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.724100    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5khrg\" (UniqueName: \"kubernetes.io/projected/795704e1-5f91-4636-a0b5-d9271dc4551d-kube-api-access-5khrg\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:49.724327 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.724110    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:49.724327 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.724119    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/795704e1-5f91-4636-a0b5-d9271dc4551d-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:49.972328 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.972238    5820 generic.go:356] "Generic (PLEG): container finished" podID="795704e1-5f91-4636-a0b5-d9271dc4551d" containerID="3369c4799f7787fdff17f62a41cb6adfc659cc0180dde0c1278fd875ee84c46b" exitCode=0
Oct 30 19:40:49.972328 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.972299    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"
Oct 30 19:40:49.972548 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.972333    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2" event={"ID":"795704e1-5f91-4636-a0b5-d9271dc4551d","Type":"ContainerDied","Data":"3369c4799f7787fdff17f62a41cb6adfc659cc0180dde0c1278fd875ee84c46b"}
Oct 30 19:40:49.972548 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.972434    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2" event={"ID":"795704e1-5f91-4636-a0b5-d9271dc4551d","Type":"ContainerDied","Data":"f4ec8f4e30728313015242448217b089ec74e528c74fa26bb1107c3ba11d7d4a"}
Oct 30 19:40:49.972548 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.972465    5820 scope.go:122] "RemoveContainer" containerID="3369c4799f7787fdff17f62a41cb6adfc659cc0180dde0c1278fd875ee84c46b"
Oct 30 19:40:49.983423 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.983405    5820 scope.go:122] "RemoveContainer" containerID="02a2552ec01a773a0fedd7dc564aedc21963c9e9e654e0a47860038b1e93382e"
Oct 30 19:40:49.992207 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.992188    5820 scope.go:122] "RemoveContainer" containerID="3369c4799f7787fdff17f62a41cb6adfc659cc0180dde0c1278fd875ee84c46b"
Oct 30 19:40:49.992504 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:40:49.992482    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3369c4799f7787fdff17f62a41cb6adfc659cc0180dde0c1278fd875ee84c46b\": container with ID starting with 3369c4799f7787fdff17f62a41cb6adfc659cc0180dde0c1278fd875ee84c46b not found: ID does not exist" containerID="3369c4799f7787fdff17f62a41cb6adfc659cc0180dde0c1278fd875ee84c46b"
Oct 30 19:40:49.992612 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.992510    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3369c4799f7787fdff17f62a41cb6adfc659cc0180dde0c1278fd875ee84c46b"} err="failed to get container status \"3369c4799f7787fdff17f62a41cb6adfc659cc0180dde0c1278fd875ee84c46b\": rpc error: code = NotFound desc = could not find container \"3369c4799f7787fdff17f62a41cb6adfc659cc0180dde0c1278fd875ee84c46b\": container with ID starting with 3369c4799f7787fdff17f62a41cb6adfc659cc0180dde0c1278fd875ee84c46b not found: ID does not exist"
Oct 30 19:40:49.992612 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.992528    5820 scope.go:122] "RemoveContainer" containerID="02a2552ec01a773a0fedd7dc564aedc21963c9e9e654e0a47860038b1e93382e"
Oct 30 19:40:49.992829 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:40:49.992764    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02a2552ec01a773a0fedd7dc564aedc21963c9e9e654e0a47860038b1e93382e\": container with ID starting with 02a2552ec01a773a0fedd7dc564aedc21963c9e9e654e0a47860038b1e93382e not found: ID does not exist" containerID="02a2552ec01a773a0fedd7dc564aedc21963c9e9e654e0a47860038b1e93382e"
Oct 30 19:40:49.992829 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:49.992785    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02a2552ec01a773a0fedd7dc564aedc21963c9e9e654e0a47860038b1e93382e"} err="failed to get container status \"02a2552ec01a773a0fedd7dc564aedc21963c9e9e654e0a47860038b1e93382e\": rpc error: code = NotFound desc = could not find container \"02a2552ec01a773a0fedd7dc564aedc21963c9e9e654e0a47860038b1e93382e\": container with ID starting with 02a2552ec01a773a0fedd7dc564aedc21963c9e9e654e0a47860038b1e93382e not found: ID does not exist"
Oct 30 19:40:50.140344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:50.140313    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"]
Oct 30 19:40:50.189657 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:50.189622    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-8f9c6b-openshift-default-7b6df7d979-cgtf2"]
Oct 30 19:40:51.436350 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:51.436321    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="795704e1-5f91-4636-a0b5-d9271dc4551d" path="/var/lib/kubelet/pods/795704e1-5f91-4636-a0b5-d9271dc4551d/volumes"
Oct 30 19:40:51.671987 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:51.671923    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:53.856041 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:53.856000    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw" podUID="bf3c4c7a-8bb7-440f-8d63-1d961dca8371" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:40:56.542204 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:56.542165    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:57.068117 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:57.067428    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:57.248005 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:40:57.247939    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:01.546070 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:01.546032    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:03.856480 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:03.856429    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw" podUID="bf3c4c7a-8bb7-440f-8d63-1d961dca8371" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:41:04.286660 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:04.286632    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw"
Oct 30 19:41:04.389580 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:04.389548    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bf3c4c7a-8bb7-440f-8d63-1d961dca8371-kube-api-access-m7c4k\" (UniqueName: \"kubernetes.io/projected/bf3c4c7a-8bb7-440f-8d63-1d961dca8371-kube-api-access-m7c4k\") pod \"bf3c4c7a-8bb7-440f-8d63-1d961dca8371\" (UID: \"bf3c4c7a-8bb7-440f-8d63-1d961dca8371\") "
Oct 30 19:41:04.389776 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:04.389663    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bf3c4c7a-8bb7-440f-8d63-1d961dca8371-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf3c4c7a-8bb7-440f-8d63-1d961dca8371-plugins-config-volume\") pod \"bf3c4c7a-8bb7-440f-8d63-1d961dca8371\" (UID: \"bf3c4c7a-8bb7-440f-8d63-1d961dca8371\") "
Oct 30 19:41:04.390079 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:04.390051    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf3c4c7a-8bb7-440f-8d63-1d961dca8371-plugins-config-volume" pod "bf3c4c7a-8bb7-440f-8d63-1d961dca8371" (UID: "bf3c4c7a-8bb7-440f-8d63-1d961dca8371"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:41:04.391848 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:04.391823    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf3c4c7a-8bb7-440f-8d63-1d961dca8371-kube-api-access-m7c4k" pod "bf3c4c7a-8bb7-440f-8d63-1d961dca8371" (UID: "bf3c4c7a-8bb7-440f-8d63-1d961dca8371"). InnerVolumeSpecName "kube-api-access-m7c4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:04.490921 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:04.490879    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7c4k\" (UniqueName: \"kubernetes.io/projected/bf3c4c7a-8bb7-440f-8d63-1d961dca8371-kube-api-access-m7c4k\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:04.490921 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:04.490911    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf3c4c7a-8bb7-440f-8d63-1d961dca8371-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:05.062496 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:05.062465    5820 generic.go:356] "Generic (PLEG): container finished" podID="bf3c4c7a-8bb7-440f-8d63-1d961dca8371" containerID="e7b981d4fc6545892f11d5ea9d889e78e423b5e5c11e9e722f232a2e55768cd0" exitCode=0
Oct 30 19:41:05.062934 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:05.062506    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw"
Oct 30 19:41:05.062934 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:05.062556    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw" event={"ID":"bf3c4c7a-8bb7-440f-8d63-1d961dca8371","Type":"ContainerDied","Data":"e7b981d4fc6545892f11d5ea9d889e78e423b5e5c11e9e722f232a2e55768cd0"}
Oct 30 19:41:05.062934 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:05.062605    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw" event={"ID":"bf3c4c7a-8bb7-440f-8d63-1d961dca8371","Type":"ContainerDied","Data":"7b5dcf4b1e985fd03e5f7f6a77b420cdda7946001c3a8a0f1e7bbee28d51255b"}
Oct 30 19:41:05.062934 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:05.062627    5820 scope.go:122] "RemoveContainer" containerID="e7b981d4fc6545892f11d5ea9d889e78e423b5e5c11e9e722f232a2e55768cd0"
Oct 30 19:41:05.072512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:05.072496    5820 scope.go:122] "RemoveContainer" containerID="e7b981d4fc6545892f11d5ea9d889e78e423b5e5c11e9e722f232a2e55768cd0"
Oct 30 19:41:05.072793 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:41:05.072770    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7b981d4fc6545892f11d5ea9d889e78e423b5e5c11e9e722f232a2e55768cd0\": container with ID starting with e7b981d4fc6545892f11d5ea9d889e78e423b5e5c11e9e722f232a2e55768cd0 not found: ID does not exist" containerID="e7b981d4fc6545892f11d5ea9d889e78e423b5e5c11e9e722f232a2e55768cd0"
Oct 30 19:41:05.072851 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:05.072803    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7b981d4fc6545892f11d5ea9d889e78e423b5e5c11e9e722f232a2e55768cd0"} err="failed to get container status \"e7b981d4fc6545892f11d5ea9d889e78e423b5e5c11e9e722f232a2e55768cd0\": rpc error: code = NotFound desc = could not find container \"e7b981d4fc6545892f11d5ea9d889e78e423b5e5c11e9e722f232a2e55768cd0\": container with ID starting with e7b981d4fc6545892f11d5ea9d889e78e423b5e5c11e9e722f232a2e55768cd0 not found: ID does not exist"
Oct 30 19:41:05.227797 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:05.227756    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw"]
Oct 30 19:41:05.276797 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:05.276761    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-23302c-b6d55547-blcvw"]
Oct 30 19:41:05.436516 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:05.436427    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bf3c4c7a-8bb7-440f-8d63-1d961dca8371" path="/var/lib/kubelet/pods/bf3c4c7a-8bb7-440f-8d63-1d961dca8371/volumes"
Oct 30 19:41:06.327209 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:06.327170    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:08.203058 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:41:08.203021    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:41:09.243675 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:09.243641    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:10.338854 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:10.338815    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"]
Oct 30 19:41:10.339376 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:10.339036    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc" podUID="19f25149-4054-4b20-8674-220824e695c3" containerName="maas-api" containerID="cri-o://6cb55ff0679e093579b0af7c9f2b8fe2f884f0a270ead55c07e6be2ac023dc2a" gracePeriod=30
Oct 30 19:41:10.486810 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:10.486770    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:10.628620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:10.628595    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"
Oct 30 19:41:10.760259 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:10.760223    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19f25149-4054-4b20-8674-220824e695c3-kube-api-access-8dvf9\" (UniqueName: \"kubernetes.io/projected/19f25149-4054-4b20-8674-220824e695c3-kube-api-access-8dvf9\") pod \"19f25149-4054-4b20-8674-220824e695c3\" (UID: \"19f25149-4054-4b20-8674-220824e695c3\") "
Oct 30 19:41:10.760473 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:10.760329    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19f25149-4054-4b20-8674-220824e695c3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19f25149-4054-4b20-8674-220824e695c3-openshift-service-ca\") pod \"19f25149-4054-4b20-8674-220824e695c3\" (UID: \"19f25149-4054-4b20-8674-220824e695c3\") "
Oct 30 19:41:10.760473 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:10.760402    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19f25149-4054-4b20-8674-220824e695c3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19f25149-4054-4b20-8674-220824e695c3-maas-api-tls\") pod \"19f25149-4054-4b20-8674-220824e695c3\" (UID: \"19f25149-4054-4b20-8674-220824e695c3\") "
Oct 30 19:41:10.760622 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:10.760514    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19f25149-4054-4b20-8674-220824e695c3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/19f25149-4054-4b20-8674-220824e695c3-metrics-certs\") pod \"19f25149-4054-4b20-8674-220824e695c3\" (UID: \"19f25149-4054-4b20-8674-220824e695c3\") "
Oct 30 19:41:10.760701 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:10.760674    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19f25149-4054-4b20-8674-220824e695c3-openshift-service-ca" pod "19f25149-4054-4b20-8674-220824e695c3" (UID: "19f25149-4054-4b20-8674-220824e695c3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:41:10.761055 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:10.761032    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19f25149-4054-4b20-8674-220824e695c3-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:10.763114 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:10.763074    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19f25149-4054-4b20-8674-220824e695c3-metrics-certs" pod "19f25149-4054-4b20-8674-220824e695c3" (UID: "19f25149-4054-4b20-8674-220824e695c3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:41:10.763226 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:10.763152    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19f25149-4054-4b20-8674-220824e695c3-maas-api-tls" pod "19f25149-4054-4b20-8674-220824e695c3" (UID: "19f25149-4054-4b20-8674-220824e695c3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:41:10.763293 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:10.763226    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19f25149-4054-4b20-8674-220824e695c3-kube-api-access-8dvf9" pod "19f25149-4054-4b20-8674-220824e695c3" (UID: "19f25149-4054-4b20-8674-220824e695c3"). InnerVolumeSpecName "kube-api-access-8dvf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:10.862618 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:10.862491    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19f25149-4054-4b20-8674-220824e695c3-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:10.862618 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:10.862527    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dvf9\" (UniqueName: \"kubernetes.io/projected/19f25149-4054-4b20-8674-220824e695c3-kube-api-access-8dvf9\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:10.862618 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:10.862542    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19f25149-4054-4b20-8674-220824e695c3-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:11.099074 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:11.099037    5820 generic.go:356] "Generic (PLEG): container finished" podID="19f25149-4054-4b20-8674-220824e695c3" containerID="6cb55ff0679e093579b0af7c9f2b8fe2f884f0a270ead55c07e6be2ac023dc2a" exitCode=0
Oct 30 19:41:11.099311 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:11.099206    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc" event={"ID":"19f25149-4054-4b20-8674-220824e695c3","Type":"ContainerDied","Data":"6cb55ff0679e093579b0af7c9f2b8fe2f884f0a270ead55c07e6be2ac023dc2a"}
Oct 30 19:41:11.099311 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:11.099249    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc" event={"ID":"19f25149-4054-4b20-8674-220824e695c3","Type":"ContainerDied","Data":"ac3c3702cd291fb22ee77ad626f666902b23c881427d6951a32e05e4bdf4f720"}
Oct 30 19:41:11.099311 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:11.099270    5820 scope.go:122] "RemoveContainer" containerID="6cb55ff0679e093579b0af7c9f2b8fe2f884f0a270ead55c07e6be2ac023dc2a"
Oct 30 19:41:11.099496 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:11.099394    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"
Oct 30 19:41:11.117847 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:11.117817    5820 scope.go:122] "RemoveContainer" containerID="6cb55ff0679e093579b0af7c9f2b8fe2f884f0a270ead55c07e6be2ac023dc2a"
Oct 30 19:41:11.118271 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:41:11.118247    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cb55ff0679e093579b0af7c9f2b8fe2f884f0a270ead55c07e6be2ac023dc2a\": container with ID starting with 6cb55ff0679e093579b0af7c9f2b8fe2f884f0a270ead55c07e6be2ac023dc2a not found: ID does not exist" containerID="6cb55ff0679e093579b0af7c9f2b8fe2f884f0a270ead55c07e6be2ac023dc2a"
Oct 30 19:41:11.118386 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:11.118280    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cb55ff0679e093579b0af7c9f2b8fe2f884f0a270ead55c07e6be2ac023dc2a"} err="failed to get container status \"6cb55ff0679e093579b0af7c9f2b8fe2f884f0a270ead55c07e6be2ac023dc2a\": rpc error: code = NotFound desc = could not find container \"6cb55ff0679e093579b0af7c9f2b8fe2f884f0a270ead55c07e6be2ac023dc2a\": container with ID starting with 6cb55ff0679e093579b0af7c9f2b8fe2f884f0a270ead55c07e6be2ac023dc2a not found: ID does not exist"
Oct 30 19:41:11.268315 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:11.268279    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"]
Oct 30 19:41:11.322932 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:11.322893    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-345670-77b78687cb-j9pnc"]
Oct 30 19:41:11.436875 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:11.436794    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="19f25149-4054-4b20-8674-220824e695c3" path="/var/lib/kubelet/pods/19f25149-4054-4b20-8674-220824e695c3/volumes"
Oct 30 19:41:11.985642 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:11.985609    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:12.200851 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:12.200804    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:12.374841 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:12.374803    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:12.891707 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:12.891676    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:17.493676 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:17.492236    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:19.444339 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:19.444298    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:19.670327 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:19.670286    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:19.792024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:19.791922    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"]
Oct 30 19:41:19.792208 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:19.792182    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd" podUID="a396603b-86a8-40ff-b353-0c9378d9f79f" containerName="istio-proxy" containerID="cri-o://60ab157a32c0fea71740d1088845ee8b716728df1f2303385e6f945d9d09576a" gracePeriod=30
Oct 30 19:41:19.870649 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:19.870606    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:22.989591 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:22.989553    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:25.064885 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.064830    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:41:25.117012 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.116980    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-data\") pod \"a396603b-86a8-40ff-b353-0c9378d9f79f\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") "
Oct 30 19:41:25.117188 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.117074    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-envoy\") pod \"a396603b-86a8-40ff-b353-0c9378d9f79f\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") "
Oct 30 19:41:25.117188 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.117149    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a396603b-86a8-40ff-b353-0c9378d9f79f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a396603b-86a8-40ff-b353-0c9378d9f79f-istiod-ca-cert\") pod \"a396603b-86a8-40ff-b353-0c9378d9f79f\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") "
Oct 30 19:41:25.117188 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.117178    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-workload-certs\") pod \"a396603b-86a8-40ff-b353-0c9378d9f79f\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") "
Oct 30 19:41:25.117406 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.117210    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a396603b-86a8-40ff-b353-0c9378d9f79f-kube-api-access-bxdbq\" (UniqueName: \"kubernetes.io/projected/a396603b-86a8-40ff-b353-0c9378d9f79f-kube-api-access-bxdbq\") pod \"a396603b-86a8-40ff-b353-0c9378d9f79f\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") "
Oct 30 19:41:25.117406 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.117245    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-token\" (UniqueName: \"kubernetes.io/projected/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-token\") pod \"a396603b-86a8-40ff-b353-0c9378d9f79f\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") "
Oct 30 19:41:25.117406 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.117333    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-credential-socket\") pod \"a396603b-86a8-40ff-b353-0c9378d9f79f\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") "
Oct 30 19:41:25.117406 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.117347    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-data" pod "a396603b-86a8-40ff-b353-0c9378d9f79f" (UID: "a396603b-86a8-40ff-b353-0c9378d9f79f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:25.117406 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.117390    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-workload-socket\") pod \"a396603b-86a8-40ff-b353-0c9378d9f79f\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") "
Oct 30 19:41:25.117743 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.117439    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-podinfo\") pod \"a396603b-86a8-40ff-b353-0c9378d9f79f\" (UID: \"a396603b-86a8-40ff-b353-0c9378d9f79f\") "
Oct 30 19:41:25.117743 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.117655    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-credential-socket" pod "a396603b-86a8-40ff-b353-0c9378d9f79f" (UID: "a396603b-86a8-40ff-b353-0c9378d9f79f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:25.117743 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.117686    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a396603b-86a8-40ff-b353-0c9378d9f79f-istiod-ca-cert" pod "a396603b-86a8-40ff-b353-0c9378d9f79f" (UID: "a396603b-86a8-40ff-b353-0c9378d9f79f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:41:25.117922 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.117856    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-workload-socket" pod "a396603b-86a8-40ff-b353-0c9378d9f79f" (UID: "a396603b-86a8-40ff-b353-0c9378d9f79f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:25.118085 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.118045    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a396603b-86a8-40ff-b353-0c9378d9f79f-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:25.118085 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.118075    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:25.118085 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.118087    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:25.118085 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.118096    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:25.119164 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.119131    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-workload-certs" pod "a396603b-86a8-40ff-b353-0c9378d9f79f" (UID: "a396603b-86a8-40ff-b353-0c9378d9f79f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:25.119771 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.119734    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-token" pod "a396603b-86a8-40ff-b353-0c9378d9f79f" (UID: "a396603b-86a8-40ff-b353-0c9378d9f79f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:25.119855 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.119819    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-envoy" pod "a396603b-86a8-40ff-b353-0c9378d9f79f" (UID: "a396603b-86a8-40ff-b353-0c9378d9f79f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:25.119919 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.119860    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a396603b-86a8-40ff-b353-0c9378d9f79f-kube-api-access-bxdbq" pod "a396603b-86a8-40ff-b353-0c9378d9f79f" (UID: "a396603b-86a8-40ff-b353-0c9378d9f79f"). InnerVolumeSpecName "kube-api-access-bxdbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:25.120355 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.120300    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-podinfo" pod "a396603b-86a8-40ff-b353-0c9378d9f79f" (UID: "a396603b-86a8-40ff-b353-0c9378d9f79f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:41:25.190593 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.190553    5820 generic.go:356] "Generic (PLEG): container finished" podID="a396603b-86a8-40ff-b353-0c9378d9f79f" containerID="60ab157a32c0fea71740d1088845ee8b716728df1f2303385e6f945d9d09576a" exitCode=0
Oct 30 19:41:25.190782 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.190618    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"
Oct 30 19:41:25.190782 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.190619    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd" event={"ID":"a396603b-86a8-40ff-b353-0c9378d9f79f","Type":"ContainerDied","Data":"60ab157a32c0fea71740d1088845ee8b716728df1f2303385e6f945d9d09576a"}
Oct 30 19:41:25.190782 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.190763    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd" event={"ID":"a396603b-86a8-40ff-b353-0c9378d9f79f","Type":"ContainerDied","Data":"5e3ca59f14b6dfa452760ca712d68432c6d21ea1ad4ba4b4183322ecc780823c"}
Oct 30 19:41:25.190986 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.190794    5820 scope.go:122] "RemoveContainer" containerID="60ab157a32c0fea71740d1088845ee8b716728df1f2303385e6f945d9d09576a"
Oct 30 19:41:25.201621 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.201594    5820 scope.go:122] "RemoveContainer" containerID="912377f1269aab1fb9550a787c78369bd46d5da877a8bb5a52f37b33c2d3ac2a"
Oct 30 19:41:25.210477 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.210460    5820 scope.go:122] "RemoveContainer" containerID="60ab157a32c0fea71740d1088845ee8b716728df1f2303385e6f945d9d09576a"
Oct 30 19:41:25.210722 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:41:25.210705    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60ab157a32c0fea71740d1088845ee8b716728df1f2303385e6f945d9d09576a\": container with ID starting with 60ab157a32c0fea71740d1088845ee8b716728df1f2303385e6f945d9d09576a not found: ID does not exist" containerID="60ab157a32c0fea71740d1088845ee8b716728df1f2303385e6f945d9d09576a"
Oct 30 19:41:25.210774 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.210731    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60ab157a32c0fea71740d1088845ee8b716728df1f2303385e6f945d9d09576a"} err="failed to get container status \"60ab157a32c0fea71740d1088845ee8b716728df1f2303385e6f945d9d09576a\": rpc error: code = NotFound desc = could not find container \"60ab157a32c0fea71740d1088845ee8b716728df1f2303385e6f945d9d09576a\": container with ID starting with 60ab157a32c0fea71740d1088845ee8b716728df1f2303385e6f945d9d09576a not found: ID does not exist"
Oct 30 19:41:25.210774 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.210749    5820 scope.go:122] "RemoveContainer" containerID="912377f1269aab1fb9550a787c78369bd46d5da877a8bb5a52f37b33c2d3ac2a"
Oct 30 19:41:25.211001 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:41:25.210977    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"912377f1269aab1fb9550a787c78369bd46d5da877a8bb5a52f37b33c2d3ac2a\": container with ID starting with 912377f1269aab1fb9550a787c78369bd46d5da877a8bb5a52f37b33c2d3ac2a not found: ID does not exist" containerID="912377f1269aab1fb9550a787c78369bd46d5da877a8bb5a52f37b33c2d3ac2a"
Oct 30 19:41:25.211078 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.211012    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"912377f1269aab1fb9550a787c78369bd46d5da877a8bb5a52f37b33c2d3ac2a"} err="failed to get container status \"912377f1269aab1fb9550a787c78369bd46d5da877a8bb5a52f37b33c2d3ac2a\": rpc error: code = NotFound desc = could not find container \"912377f1269aab1fb9550a787c78369bd46d5da877a8bb5a52f37b33c2d3ac2a\": container with ID starting with 912377f1269aab1fb9550a787c78369bd46d5da877a8bb5a52f37b33c2d3ac2a not found: ID does not exist"
Oct 30 19:41:25.219160 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.219137    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:25.219160 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.219161    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:25.219296 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.219171    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a396603b-86a8-40ff-b353-0c9378d9f79f-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:25.219296 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.219181    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxdbq\" (UniqueName: \"kubernetes.io/projected/a396603b-86a8-40ff-b353-0c9378d9f79f-kube-api-access-bxdbq\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:25.219296 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.219190    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a396603b-86a8-40ff-b353-0c9378d9f79f-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:25.347865 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.347780    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"]
Oct 30 19:41:25.352570 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.352542    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-345670-openshift-default-6f4f89f789-5wjpd"]
Oct 30 19:41:25.437347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:25.437316    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a396603b-86a8-40ff-b353-0c9378d9f79f" path="/var/lib/kubelet/pods/a396603b-86a8-40ff-b353-0c9378d9f79f/volumes"
Oct 30 19:41:27.988330 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:27.988292    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:32.014256 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:32.014209    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"]
Oct 30 19:41:32.014769 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:32.014530    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5" podUID="18dfcf17-bc5a-43d9-aff6-4cc57d2a1233" containerName="maas-api" containerID="cri-o://f1da6cce9f665398da85f3a603d674e9a647b6715e76b41ca0212055b43a1db8" gracePeriod=30
Oct 30 19:41:32.169846 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:32.169808    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:32.245169 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:32.245128    5820 generic.go:356] "Generic (PLEG): container finished" podID="18dfcf17-bc5a-43d9-aff6-4cc57d2a1233" containerID="f1da6cce9f665398da85f3a603d674e9a647b6715e76b41ca0212055b43a1db8" exitCode=0
Oct 30 19:41:32.245362 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:32.245294    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5" event={"ID":"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233","Type":"ContainerDied","Data":"f1da6cce9f665398da85f3a603d674e9a647b6715e76b41ca0212055b43a1db8"}
Oct 30 19:41:32.303485 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:32.303460    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"
Oct 30 19:41:32.402999 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:32.401685    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-metrics-certs\" (UniqueName: \"kubernetes.io/secret/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-metrics-certs\") pod \"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233\" (UID: \"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233\") "
Oct 30 19:41:32.402999 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:32.402082    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-maas-api-tls\") pod \"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233\" (UID: \"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233\") "
Oct 30 19:41:32.402999 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:32.402635    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-openshift-service-ca\") pod \"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233\" (UID: \"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233\") "
Oct 30 19:41:32.402999 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:32.402678    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-kube-api-access-vrs9c\" (UniqueName: \"kubernetes.io/projected/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-kube-api-access-vrs9c\") pod \"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233\" (UID: \"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233\") "
Oct 30 19:41:32.403375 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:32.403101    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-openshift-service-ca" pod "18dfcf17-bc5a-43d9-aff6-4cc57d2a1233" (UID: "18dfcf17-bc5a-43d9-aff6-4cc57d2a1233"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:41:32.403375 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:32.403261    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:32.404713 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:32.404680    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-metrics-certs" pod "18dfcf17-bc5a-43d9-aff6-4cc57d2a1233" (UID: "18dfcf17-bc5a-43d9-aff6-4cc57d2a1233"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:41:32.405633 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:32.405583    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-kube-api-access-vrs9c" pod "18dfcf17-bc5a-43d9-aff6-4cc57d2a1233" (UID: "18dfcf17-bc5a-43d9-aff6-4cc57d2a1233"). InnerVolumeSpecName "kube-api-access-vrs9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:32.407019 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:32.406035    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-maas-api-tls" pod "18dfcf17-bc5a-43d9-aff6-4cc57d2a1233" (UID: "18dfcf17-bc5a-43d9-aff6-4cc57d2a1233"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:41:32.504047 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:32.504014    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:32.504047 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:32.504040    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrs9c\" (UniqueName: \"kubernetes.io/projected/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-kube-api-access-vrs9c\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:32.504301 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:32.504055    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:33.183141 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.183107    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"]
Oct 30 19:41:33.183589 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.183316    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq" podUID="85a27989-15d4-4c57-bdcd-1ce98d0811b2" containerName="maas-api" containerID="cri-o://e4d73e22f3fd18d7c2524d1cd092d946f8752425126bf5f002f6e3b318fa4aa2" gracePeriod=30
Oct 30 19:41:33.253087 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.253054    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"
Oct 30 19:41:33.253087 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.253072    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5" event={"ID":"18dfcf17-bc5a-43d9-aff6-4cc57d2a1233","Type":"ContainerDied","Data":"ad2d7acd72db360825371bb7977ff9679ffcc09dbf435cbaf2e3c32afb6bf7ad"}
Oct 30 19:41:33.253373 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.253139    5820 scope.go:122] "RemoveContainer" containerID="f1da6cce9f665398da85f3a603d674e9a647b6715e76b41ca0212055b43a1db8"
Oct 30 19:41:33.333299 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.333258    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:33.420428 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.414837    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"]
Oct 30 19:41:33.420678 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.420650    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-23302c-ff9b97b94-76qm5"]
Oct 30 19:41:33.436942 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.436868    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="18dfcf17-bc5a-43d9-aff6-4cc57d2a1233" path="/var/lib/kubelet/pods/18dfcf17-bc5a-43d9-aff6-4cc57d2a1233/volumes"
Oct 30 19:41:33.473235 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.473211    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"
Oct 30 19:41:33.515577 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.515537    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85a27989-15d4-4c57-bdcd-1ce98d0811b2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85a27989-15d4-4c57-bdcd-1ce98d0811b2-maas-api-tls\") pod \"85a27989-15d4-4c57-bdcd-1ce98d0811b2\" (UID: \"85a27989-15d4-4c57-bdcd-1ce98d0811b2\") "
Oct 30 19:41:33.515704 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.515590    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85a27989-15d4-4c57-bdcd-1ce98d0811b2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85a27989-15d4-4c57-bdcd-1ce98d0811b2-openshift-service-ca\") pod \"85a27989-15d4-4c57-bdcd-1ce98d0811b2\" (UID: \"85a27989-15d4-4c57-bdcd-1ce98d0811b2\") "
Oct 30 19:41:33.515773 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.515745    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85a27989-15d4-4c57-bdcd-1ce98d0811b2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/85a27989-15d4-4c57-bdcd-1ce98d0811b2-metrics-certs\") pod \"85a27989-15d4-4c57-bdcd-1ce98d0811b2\" (UID: \"85a27989-15d4-4c57-bdcd-1ce98d0811b2\") "
Oct 30 19:41:33.515867 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.515849    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85a27989-15d4-4c57-bdcd-1ce98d0811b2-kube-api-access-gzg9c\" (UniqueName: \"kubernetes.io/projected/85a27989-15d4-4c57-bdcd-1ce98d0811b2-kube-api-access-gzg9c\") pod \"85a27989-15d4-4c57-bdcd-1ce98d0811b2\" (UID: \"85a27989-15d4-4c57-bdcd-1ce98d0811b2\") "
Oct 30 19:41:33.516000 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.515940    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85a27989-15d4-4c57-bdcd-1ce98d0811b2-openshift-service-ca" pod "85a27989-15d4-4c57-bdcd-1ce98d0811b2" (UID: "85a27989-15d4-4c57-bdcd-1ce98d0811b2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:41:33.516434 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.516408    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85a27989-15d4-4c57-bdcd-1ce98d0811b2-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:33.518166 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.518139    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85a27989-15d4-4c57-bdcd-1ce98d0811b2-kube-api-access-gzg9c" pod "85a27989-15d4-4c57-bdcd-1ce98d0811b2" (UID: "85a27989-15d4-4c57-bdcd-1ce98d0811b2"). InnerVolumeSpecName "kube-api-access-gzg9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:33.518489 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.518472    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85a27989-15d4-4c57-bdcd-1ce98d0811b2-maas-api-tls" pod "85a27989-15d4-4c57-bdcd-1ce98d0811b2" (UID: "85a27989-15d4-4c57-bdcd-1ce98d0811b2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:41:33.518556 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.518496    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85a27989-15d4-4c57-bdcd-1ce98d0811b2-metrics-certs" pod "85a27989-15d4-4c57-bdcd-1ce98d0811b2" (UID: "85a27989-15d4-4c57-bdcd-1ce98d0811b2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:41:33.617556 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.617524    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85a27989-15d4-4c57-bdcd-1ce98d0811b2-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:33.617556 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.617555    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzg9c\" (UniqueName: \"kubernetes.io/projected/85a27989-15d4-4c57-bdcd-1ce98d0811b2-kube-api-access-gzg9c\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:33.617756 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.617567    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85a27989-15d4-4c57-bdcd-1ce98d0811b2-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:33.701737 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:33.701646    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:34.060330 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:34.060296    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:34.260094 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:34.260059    5820 generic.go:356] "Generic (PLEG): container finished" podID="85a27989-15d4-4c57-bdcd-1ce98d0811b2" containerID="e4d73e22f3fd18d7c2524d1cd092d946f8752425126bf5f002f6e3b318fa4aa2" exitCode=0
Oct 30 19:41:34.260588 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:34.260104    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"
Oct 30 19:41:34.260588 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:34.260140    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq" event={"ID":"85a27989-15d4-4c57-bdcd-1ce98d0811b2","Type":"ContainerDied","Data":"e4d73e22f3fd18d7c2524d1cd092d946f8752425126bf5f002f6e3b318fa4aa2"}
Oct 30 19:41:34.260588 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:34.260200    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq" event={"ID":"85a27989-15d4-4c57-bdcd-1ce98d0811b2","Type":"ContainerDied","Data":"a67bdfa1bb8d387825ec94555907ce35f6145942e71e910f5925253e498abed0"}
Oct 30 19:41:34.260588 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:34.260237    5820 scope.go:122] "RemoveContainer" containerID="e4d73e22f3fd18d7c2524d1cd092d946f8752425126bf5f002f6e3b318fa4aa2"
Oct 30 19:41:34.270615 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:34.270597    5820 scope.go:122] "RemoveContainer" containerID="e4d73e22f3fd18d7c2524d1cd092d946f8752425126bf5f002f6e3b318fa4aa2"
Oct 30 19:41:34.270968 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:41:34.270931    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4d73e22f3fd18d7c2524d1cd092d946f8752425126bf5f002f6e3b318fa4aa2\": container with ID starting with e4d73e22f3fd18d7c2524d1cd092d946f8752425126bf5f002f6e3b318fa4aa2 not found: ID does not exist" containerID="e4d73e22f3fd18d7c2524d1cd092d946f8752425126bf5f002f6e3b318fa4aa2"
Oct 30 19:41:34.271063 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:34.270982    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4d73e22f3fd18d7c2524d1cd092d946f8752425126bf5f002f6e3b318fa4aa2"} err="failed to get container status \"e4d73e22f3fd18d7c2524d1cd092d946f8752425126bf5f002f6e3b318fa4aa2\": rpc error: code = NotFound desc = could not find container \"e4d73e22f3fd18d7c2524d1cd092d946f8752425126bf5f002f6e3b318fa4aa2\": container with ID starting with e4d73e22f3fd18d7c2524d1cd092d946f8752425126bf5f002f6e3b318fa4aa2 not found: ID does not exist"
Oct 30 19:41:34.451536 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:34.451456    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"]
Oct 30 19:41:34.500468 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:34.500437    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-dc5cdd-5c74c56f94-cktmq"]
Oct 30 19:41:34.779434 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:34.779348    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:34.963657 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:34.963626    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:35.165004 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:35.164935    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:35.335757 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:35.335722    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:35.436645 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:35.436563    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="85a27989-15d4-4c57-bdcd-1ce98d0811b2" path="/var/lib/kubelet/pods/85a27989-15d4-4c57-bdcd-1ce98d0811b2/volumes"
Oct 30 19:41:35.967393 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:35.967356    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:37.190091 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:37.190051    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:38.202305 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:41:38.202270    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:41:38.625620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:38.625569    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:39.352801 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:39.352767    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:39.548025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:39.547982    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:40.450401 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:40.450320    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:42.603420 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:42.603380    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:42.793967 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:42.793919    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:42.937163 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:42.937084    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"]
Oct 30 19:41:42.937331 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:42.937305    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt" podUID="f4415a3e-5238-48e7-a6b2-e97c1904381f" containerName="istio-proxy" containerID="cri-o://679e07438b23f5986fe508afd5d9ae4ae6f4d76a2273143e772749cb59fca21f" gracePeriod=30
Oct 30 19:41:42.982025 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:42.981989    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:43.469706 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:43.469668    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:43.644043 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:43.644002    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:43.833991 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:43.833923    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:44.318723 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:44.318686    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:44.509698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:44.509662    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:44.685249 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:44.685146    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:45.738744 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:45.738709    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:47.624862 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:47.624822    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:47.847891 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:47.847861    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:48.006930 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.006901    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw"]
Oct 30 19:41:48.007425 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007404    5820 cpu_manager.go:414] "containerMap: removing container" podUID="a396603b-86a8-40ff-b353-0c9378d9f79f" containerName="istio-proxy"
Oct 30 19:41:48.007425 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007421    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a396603b-86a8-40ff-b353-0c9378d9f79f" containerName="istio-proxy"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007433    5820 cpu_manager.go:414] "containerMap: removing container" podUID="bf3c4c7a-8bb7-440f-8d63-1d961dca8371" containerName="payload-pre-processing"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007439    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf3c4c7a-8bb7-440f-8d63-1d961dca8371" containerName="payload-pre-processing"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007451    5820 cpu_manager.go:414] "containerMap: removing container" podUID="664f1334-638a-4581-9c84-5a2ea290ec81" containerName="istio-proxy"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007458    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="664f1334-638a-4581-9c84-5a2ea290ec81" containerName="istio-proxy"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007471    5820 cpu_manager.go:414] "containerMap: removing container" podUID="85a27989-15d4-4c57-bdcd-1ce98d0811b2" containerName="maas-api"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007476    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85a27989-15d4-4c57-bdcd-1ce98d0811b2" containerName="maas-api"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007486    5820 cpu_manager.go:414] "containerMap: removing container" podUID="336a57c2-7270-4cd3-843f-b8a63a26bcab" containerName="payload-pre-processing"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007491    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="336a57c2-7270-4cd3-843f-b8a63a26bcab" containerName="payload-pre-processing"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007498    5820 cpu_manager.go:414] "containerMap: removing container" podUID="af249134-3666-43cf-b772-8178f036a60d" containerName="maas-api"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007502    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af249134-3666-43cf-b772-8178f036a60d" containerName="maas-api"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007508    5820 cpu_manager.go:414] "containerMap: removing container" podUID="a5f8e8dd-a036-477e-ac6e-b2f15500af33" containerName="maas-api"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007512    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5f8e8dd-a036-477e-ac6e-b2f15500af33" containerName="maas-api"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007525    5820 cpu_manager.go:414] "containerMap: removing container" podUID="349f7a4a-cae3-41b7-a6dd-ead82d9b0a64" containerName="payload-processing"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007530    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="349f7a4a-cae3-41b7-a6dd-ead82d9b0a64" containerName="payload-processing"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007536    5820 cpu_manager.go:414] "containerMap: removing container" podUID="19f25149-4054-4b20-8674-220824e695c3" containerName="maas-api"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007541    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19f25149-4054-4b20-8674-220824e695c3" containerName="maas-api"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007551    5820 cpu_manager.go:414] "containerMap: removing container" podUID="1ad9d9e9-639f-49a3-a53b-e0dda71e87b4" containerName="maas-api"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007555    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ad9d9e9-639f-49a3-a53b-e0dda71e87b4" containerName="maas-api"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007563    5820 cpu_manager.go:414] "containerMap: removing container" podUID="075517b2-2a7e-4edc-8658-677e52abc0c6" containerName="istio-proxy"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007567    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="075517b2-2a7e-4edc-8658-677e52abc0c6" containerName="istio-proxy"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007574    5820 cpu_manager.go:414] "containerMap: removing container" podUID="612e60bf-47da-407a-b30d-6213966bd7a9" containerName="maas-api"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007578    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="612e60bf-47da-407a-b30d-6213966bd7a9" containerName="maas-api"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007585    5820 cpu_manager.go:414] "containerMap: removing container" podUID="795704e1-5f91-4636-a0b5-d9271dc4551d" containerName="istio-proxy"
Oct 30 19:41:48.007581 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007589    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="795704e1-5f91-4636-a0b5-d9271dc4551d" containerName="istio-proxy"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007597    5820 cpu_manager.go:414] "containerMap: removing container" podUID="f66409e5-083e-4ae1-bc9a-02d1a4fd12b8" containerName="maas-api"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007602    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f66409e5-083e-4ae1-bc9a-02d1a4fd12b8" containerName="maas-api"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007615    5820 cpu_manager.go:414] "containerMap: removing container" podUID="18dfcf17-bc5a-43d9-aff6-4cc57d2a1233" containerName="maas-api"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007619    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="18dfcf17-bc5a-43d9-aff6-4cc57d2a1233" containerName="maas-api"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007625    5820 cpu_manager.go:414] "containerMap: removing container" podUID="795704e1-5f91-4636-a0b5-d9271dc4551d" containerName="istio-proxy"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007629    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="795704e1-5f91-4636-a0b5-d9271dc4551d" containerName="istio-proxy"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007636    5820 cpu_manager.go:414] "containerMap: removing container" podUID="c6c5724f-a1ce-448e-9535-3bbc49743b18" containerName="istio-proxy"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007640    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6c5724f-a1ce-448e-9535-3bbc49743b18" containerName="istio-proxy"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007646    5820 cpu_manager.go:414] "containerMap: removing container" podUID="075517b2-2a7e-4edc-8658-677e52abc0c6" containerName="istio-proxy"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007650    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="075517b2-2a7e-4edc-8658-677e52abc0c6" containerName="istio-proxy"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007657    5820 cpu_manager.go:414] "containerMap: removing container" podUID="c6c5724f-a1ce-448e-9535-3bbc49743b18" containerName="istio-proxy"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007661    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6c5724f-a1ce-448e-9535-3bbc49743b18" containerName="istio-proxy"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007727    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="18dfcf17-bc5a-43d9-aff6-4cc57d2a1233" containerName="maas-api"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007736    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="795704e1-5f91-4636-a0b5-d9271dc4551d" containerName="istio-proxy"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007742    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6c5724f-a1ce-448e-9535-3bbc49743b18" containerName="istio-proxy"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007747    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="664f1334-638a-4581-9c84-5a2ea290ec81" containerName="istio-proxy"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007752    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="075517b2-2a7e-4edc-8658-677e52abc0c6" containerName="istio-proxy"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007756    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="a396603b-86a8-40ff-b353-0c9378d9f79f" containerName="istio-proxy"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007761    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6c5724f-a1ce-448e-9535-3bbc49743b18" containerName="istio-proxy"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007767    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="a396603b-86a8-40ff-b353-0c9378d9f79f" containerName="istio-proxy"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007772    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="bf3c4c7a-8bb7-440f-8d63-1d961dca8371" containerName="payload-pre-processing"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007778    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="664f1334-638a-4581-9c84-5a2ea290ec81" containerName="istio-proxy"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007783    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="85a27989-15d4-4c57-bdcd-1ce98d0811b2" containerName="maas-api"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007789    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="336a57c2-7270-4cd3-843f-b8a63a26bcab" containerName="payload-pre-processing"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007794    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="af249134-3666-43cf-b772-8178f036a60d" containerName="maas-api"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007798    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5f8e8dd-a036-477e-ac6e-b2f15500af33" containerName="maas-api"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007803    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="349f7a4a-cae3-41b7-a6dd-ead82d9b0a64" containerName="payload-processing"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007808    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="19f25149-4054-4b20-8674-220824e695c3" containerName="maas-api"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007814    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ad9d9e9-639f-49a3-a53b-e0dda71e87b4" containerName="maas-api"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007819    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="612e60bf-47da-407a-b30d-6213966bd7a9" containerName="maas-api"
Oct 30 19:41:48.008396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.007824    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="f66409e5-083e-4ae1-bc9a-02d1a4fd12b8" containerName="maas-api"
Oct 30 19:41:48.011037 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.011021    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw"
Oct 30 19:41:48.063258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.063236    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-5da1ae\""
Oct 30 19:41:48.114184 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.114138    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-5da1ae-dockercfg-sk92r\""
Oct 30 19:41:48.115084 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.115056    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw"]
Oct 30 19:41:48.164728 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.164698    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw"]
Oct 30 19:41:48.165321 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.165306    5820 cpu_manager.go:414] "containerMap: removing container" podUID="a396603b-86a8-40ff-b353-0c9378d9f79f" containerName="istio-proxy"
Oct 30 19:41:48.165382 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.165323    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a396603b-86a8-40ff-b353-0c9378d9f79f" containerName="istio-proxy"
Oct 30 19:41:48.165382 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.165334    5820 cpu_manager.go:414] "containerMap: removing container" podUID="664f1334-638a-4581-9c84-5a2ea290ec81" containerName="istio-proxy"
Oct 30 19:41:48.165382 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.165340    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="664f1334-638a-4581-9c84-5a2ea290ec81" containerName="istio-proxy"
Oct 30 19:41:48.165487 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.165442    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="795704e1-5f91-4636-a0b5-d9271dc4551d" containerName="istio-proxy"
Oct 30 19:41:48.165487 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.165452    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="075517b2-2a7e-4edc-8658-677e52abc0c6" containerName="istio-proxy"
Oct 30 19:41:48.168742 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.168725    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw"
Oct 30 19:41:48.195295 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.195274    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d89bh\" (UniqueName: \"kubernetes.io/projected/6211ab6d-429e-41e9-b0a0-3a1d76043165-kube-api-access-d89bh\") pod \"payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw\" (UID: \"6211ab6d-429e-41e9-b0a0-3a1d76043165\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw"
Oct 30 19:41:48.195375 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.195335    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lh58\" (UniqueName: \"kubernetes.io/projected/420b7e12-f43e-45e0-be09-042d57313f5b-kube-api-access-4lh58\") pod \"payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw\" (UID: \"420b7e12-f43e-45e0-be09-042d57313f5b\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw"
Oct 30 19:41:48.195465 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.195443    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6211ab6d-429e-41e9-b0a0-3a1d76043165-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw\" (UID: \"6211ab6d-429e-41e9-b0a0-3a1d76043165\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw"
Oct 30 19:41:48.195531 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.195502    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/420b7e12-f43e-45e0-be09-042d57313f5b-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw\" (UID: \"420b7e12-f43e-45e0-be09-042d57313f5b\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw"
Oct 30 19:41:48.206287 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.206226    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw"]
Oct 30 19:41:48.215698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.215677    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:41:48.296073 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.296048    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f4415a3e-5238-48e7-a6b2-e97c1904381f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4415a3e-5238-48e7-a6b2-e97c1904381f-istiod-ca-cert\") pod \"f4415a3e-5238-48e7-a6b2-e97c1904381f\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") "
Oct 30 19:41:48.296219 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.296123    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-podinfo\") pod \"f4415a3e-5238-48e7-a6b2-e97c1904381f\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") "
Oct 30 19:41:48.296219 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.296169    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-workload-socket\") pod \"f4415a3e-5238-48e7-a6b2-e97c1904381f\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") "
Oct 30 19:41:48.296388 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.296358    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-workload-certs\") pod \"f4415a3e-5238-48e7-a6b2-e97c1904381f\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") "
Oct 30 19:41:48.296438 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.296422    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-credential-socket\") pod \"f4415a3e-5238-48e7-a6b2-e97c1904381f\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") "
Oct 30 19:41:48.296482 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.296441    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-workload-socket" pod "f4415a3e-5238-48e7-a6b2-e97c1904381f" (UID: "f4415a3e-5238-48e7-a6b2-e97c1904381f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:48.296482 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.296441    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4415a3e-5238-48e7-a6b2-e97c1904381f-istiod-ca-cert" pod "f4415a3e-5238-48e7-a6b2-e97c1904381f" (UID: "f4415a3e-5238-48e7-a6b2-e97c1904381f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:41:48.296570 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.296486    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-envoy\") pod \"f4415a3e-5238-48e7-a6b2-e97c1904381f\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") "
Oct 30 19:41:48.296570 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.296527    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4415a3e-5238-48e7-a6b2-e97c1904381f-kube-api-access-dhsx7\" (UniqueName: \"kubernetes.io/projected/f4415a3e-5238-48e7-a6b2-e97c1904381f-kube-api-access-dhsx7\") pod \"f4415a3e-5238-48e7-a6b2-e97c1904381f\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") "
Oct 30 19:41:48.296672 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.296595    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-workload-certs" pod "f4415a3e-5238-48e7-a6b2-e97c1904381f" (UID: "f4415a3e-5238-48e7-a6b2-e97c1904381f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:48.296672 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.296614    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-token\" (UniqueName: \"kubernetes.io/projected/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-token\") pod \"f4415a3e-5238-48e7-a6b2-e97c1904381f\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") "
Oct 30 19:41:48.296672 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.296648    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-data\") pod \"f4415a3e-5238-48e7-a6b2-e97c1904381f\" (UID: \"f4415a3e-5238-48e7-a6b2-e97c1904381f\") "
Oct 30 19:41:48.296854 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.296692    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-credential-socket" pod "f4415a3e-5238-48e7-a6b2-e97c1904381f" (UID: "f4415a3e-5238-48e7-a6b2-e97c1904381f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:48.296914 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.296867    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lh58\" (UniqueName: \"kubernetes.io/projected/420b7e12-f43e-45e0-be09-042d57313f5b-kube-api-access-4lh58\") pod \"payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw\" (UID: \"420b7e12-f43e-45e0-be09-042d57313f5b\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw"
Oct 30 19:41:48.297018 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.296927    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6211ab6d-429e-41e9-b0a0-3a1d76043165-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw\" (UID: \"6211ab6d-429e-41e9-b0a0-3a1d76043165\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw"
Oct 30 19:41:48.297018 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.296976    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-data" pod "f4415a3e-5238-48e7-a6b2-e97c1904381f" (UID: "f4415a3e-5238-48e7-a6b2-e97c1904381f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:48.297142 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.297037    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/420b7e12-f43e-45e0-be09-042d57313f5b-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw\" (UID: \"420b7e12-f43e-45e0-be09-042d57313f5b\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw"
Oct 30 19:41:48.297229 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.297207    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d89bh\" (UniqueName: \"kubernetes.io/projected/6211ab6d-429e-41e9-b0a0-3a1d76043165-kube-api-access-d89bh\") pod \"payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw\" (UID: \"6211ab6d-429e-41e9-b0a0-3a1d76043165\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw"
Oct 30 19:41:48.297495 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.297440    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:48.297495 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.297460    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:48.297495 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.297475    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:48.297495 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.297488    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:48.297734 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.297504    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4415a3e-5238-48e7-a6b2-e97c1904381f-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:48.297875 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.297847    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6211ab6d-429e-41e9-b0a0-3a1d76043165-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw\" (UID: \"6211ab6d-429e-41e9-b0a0-3a1d76043165\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw"
Oct 30 19:41:48.298281 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.298254    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/420b7e12-f43e-45e0-be09-042d57313f5b-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw\" (UID: \"420b7e12-f43e-45e0-be09-042d57313f5b\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw"
Oct 30 19:41:48.298702 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.298682    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-podinfo" pod "f4415a3e-5238-48e7-a6b2-e97c1904381f" (UID: "f4415a3e-5238-48e7-a6b2-e97c1904381f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:41:48.299042 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.299019    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-token" pod "f4415a3e-5238-48e7-a6b2-e97c1904381f" (UID: "f4415a3e-5238-48e7-a6b2-e97c1904381f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:48.299123 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.299062    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-envoy" pod "f4415a3e-5238-48e7-a6b2-e97c1904381f" (UID: "f4415a3e-5238-48e7-a6b2-e97c1904381f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:48.299123 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.299099    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4415a3e-5238-48e7-a6b2-e97c1904381f-kube-api-access-dhsx7" pod "f4415a3e-5238-48e7-a6b2-e97c1904381f" (UID: "f4415a3e-5238-48e7-a6b2-e97c1904381f"). InnerVolumeSpecName "kube-api-access-dhsx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:48.344249 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.344221    5820 generic.go:356] "Generic (PLEG): container finished" podID="f4415a3e-5238-48e7-a6b2-e97c1904381f" containerID="679e07438b23f5986fe508afd5d9ae4ae6f4d76a2273143e772749cb59fca21f" exitCode=0
Oct 30 19:41:48.344337 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.344278    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"
Oct 30 19:41:48.344404 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.344277    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt" event={"ID":"f4415a3e-5238-48e7-a6b2-e97c1904381f","Type":"ContainerDied","Data":"679e07438b23f5986fe508afd5d9ae4ae6f4d76a2273143e772749cb59fca21f"}
Oct 30 19:41:48.344453 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.344429    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt" event={"ID":"f4415a3e-5238-48e7-a6b2-e97c1904381f","Type":"ContainerDied","Data":"d55495bccff50b7d5a0f136c4b892345d85e5eb3d7be39f2a903851cf346408a"}
Oct 30 19:41:48.344494 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.344477    5820 scope.go:122] "RemoveContainer" containerID="679e07438b23f5986fe508afd5d9ae4ae6f4d76a2273143e772749cb59fca21f"
Oct 30 19:41:48.354414 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.354394    5820 scope.go:122] "RemoveContainer" containerID="d0721ee98511840ec85e512dbce762684b8399067cea19a5a2054635e6c3eb53"
Oct 30 19:41:48.359369 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.359352    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lh58\" (UniqueName: \"kubernetes.io/projected/420b7e12-f43e-45e0-be09-042d57313f5b-kube-api-access-4lh58\") pod \"payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw\" (UID: \"420b7e12-f43e-45e0-be09-042d57313f5b\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw"
Oct 30 19:41:48.359456 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.359436    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d89bh\" (UniqueName: \"kubernetes.io/projected/6211ab6d-429e-41e9-b0a0-3a1d76043165-kube-api-access-d89bh\") pod \"payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw\" (UID: \"6211ab6d-429e-41e9-b0a0-3a1d76043165\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw"
Oct 30 19:41:48.363374 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.363358    5820 scope.go:122] "RemoveContainer" containerID="679e07438b23f5986fe508afd5d9ae4ae6f4d76a2273143e772749cb59fca21f"
Oct 30 19:41:48.363615 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:41:48.363597    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"679e07438b23f5986fe508afd5d9ae4ae6f4d76a2273143e772749cb59fca21f\": container with ID starting with 679e07438b23f5986fe508afd5d9ae4ae6f4d76a2273143e772749cb59fca21f not found: ID does not exist" containerID="679e07438b23f5986fe508afd5d9ae4ae6f4d76a2273143e772749cb59fca21f"
Oct 30 19:41:48.363663 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.363621    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"679e07438b23f5986fe508afd5d9ae4ae6f4d76a2273143e772749cb59fca21f"} err="failed to get container status \"679e07438b23f5986fe508afd5d9ae4ae6f4d76a2273143e772749cb59fca21f\": rpc error: code = NotFound desc = could not find container \"679e07438b23f5986fe508afd5d9ae4ae6f4d76a2273143e772749cb59fca21f\": container with ID starting with 679e07438b23f5986fe508afd5d9ae4ae6f4d76a2273143e772749cb59fca21f not found: ID does not exist"
Oct 30 19:41:48.363663 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.363637    5820 scope.go:122] "RemoveContainer" containerID="d0721ee98511840ec85e512dbce762684b8399067cea19a5a2054635e6c3eb53"
Oct 30 19:41:48.363877 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:41:48.363859    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0721ee98511840ec85e512dbce762684b8399067cea19a5a2054635e6c3eb53\": container with ID starting with d0721ee98511840ec85e512dbce762684b8399067cea19a5a2054635e6c3eb53 not found: ID does not exist" containerID="d0721ee98511840ec85e512dbce762684b8399067cea19a5a2054635e6c3eb53"
Oct 30 19:41:48.363934 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.363887    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0721ee98511840ec85e512dbce762684b8399067cea19a5a2054635e6c3eb53"} err="failed to get container status \"d0721ee98511840ec85e512dbce762684b8399067cea19a5a2054635e6c3eb53\": rpc error: code = NotFound desc = could not find container \"d0721ee98511840ec85e512dbce762684b8399067cea19a5a2054635e6c3eb53\": container with ID starting with d0721ee98511840ec85e512dbce762684b8399067cea19a5a2054635e6c3eb53 not found: ID does not exist"
Oct 30 19:41:48.398616 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.398585    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:48.398616 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.398612    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:48.398751 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.398628    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhsx7\" (UniqueName: \"kubernetes.io/projected/f4415a3e-5238-48e7-a6b2-e97c1904381f-kube-api-access-dhsx7\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:48.398751 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.398642    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4415a3e-5238-48e7-a6b2-e97c1904381f-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:48.486020 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.485942    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw"
Oct 30 19:41:48.521183 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.521108    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"]
Oct 30 19:41:48.524973 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.523472    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-23302c-openshift-default-594dc778ff-4djzt"]
Oct 30 19:41:48.633603 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.633572    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw"
Oct 30 19:41:48.864591 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:48.864566    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw"]
Oct 30 19:41:48.867633 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:41:48.867608    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod420b7e12_f43e_45e0_be09_042d57313f5b.slice/crio-e626d818a2b53e82d47cb316ff7a8b874d0066e31c9df84aebfcaf4425ce1a2d WatchSource:0}: Error finding container e626d818a2b53e82d47cb316ff7a8b874d0066e31c9df84aebfcaf4425ce1a2d: Status 404 returned error can't find the container with id e626d818a2b53e82d47cb316ff7a8b874d0066e31c9df84aebfcaf4425ce1a2d
Oct 30 19:41:49.013812 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:49.013784    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw"]
Oct 30 19:41:49.015851 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:41:49.015816    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6211ab6d_429e_41e9_b0a0_3a1d76043165.slice/crio-fbcb096891ae94a47cc2eac6bde089945114bdf2b17a5d3741f4215ca53ec041 WatchSource:0}: Error finding container fbcb096891ae94a47cc2eac6bde089945114bdf2b17a5d3741f4215ca53ec041: Status 404 returned error can't find the container with id fbcb096891ae94a47cc2eac6bde089945114bdf2b17a5d3741f4215ca53ec041
Oct 30 19:41:49.261153 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:49.261073    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"]
Oct 30 19:41:49.266034 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:49.266008    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"]
Oct 30 19:41:49.351135 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:49.351094    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw" event={"ID":"420b7e12-f43e-45e0-be09-042d57313f5b","Type":"ContainerStarted","Data":"526bdff887834236f7b34d96a7d1f48eb2feea0d4f73cd8615863fe75a894919"}
Oct 30 19:41:49.351350 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:49.351154    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw" event={"ID":"420b7e12-f43e-45e0-be09-042d57313f5b","Type":"ContainerStarted","Data":"e626d818a2b53e82d47cb316ff7a8b874d0066e31c9df84aebfcaf4425ce1a2d"}
Oct 30 19:41:49.351350 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:49.351179    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw"
Oct 30 19:41:49.352819 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:49.352795    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw" event={"ID":"6211ab6d-429e-41e9-b0a0-3a1d76043165","Type":"ContainerStarted","Data":"c9b3f6b82afb57b0eb1faa281b1e25ce512b28a3a21dd09a7d320d3bdd339887"}
Oct 30 19:41:49.352943 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:49.352845    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw"
Oct 30 19:41:49.352943 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:49.352865    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw" event={"ID":"6211ab6d-429e-41e9-b0a0-3a1d76043165","Type":"ContainerStarted","Data":"fbcb096891ae94a47cc2eac6bde089945114bdf2b17a5d3741f4215ca53ec041"}
Oct 30 19:41:49.436762 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:49.436729    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f4415a3e-5238-48e7-a6b2-e97c1904381f" path="/var/lib/kubelet/pods/f4415a3e-5238-48e7-a6b2-e97c1904381f/volumes"
Oct 30 19:41:49.461220 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:49.461192    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:49.502517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:49.502152    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw" podStartSLOduration=2.502135161 podStartE2EDuration="2.502135161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:41:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:41:49.501846287 +0000 UTC m=+4030.213594983" watchObservedRunningTime="2026-10-30 19:41:49.502135161 +0000 UTC m=+4030.213883860"
Oct 30 19:41:49.621085 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:49.621013    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw" podStartSLOduration=1.620993238 podStartE2EDuration="1.620993238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:41:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:41:49.618844959 +0000 UTC m=+4030.330593657" watchObservedRunningTime="2026-10-30 19:41:49.620993238 +0000 UTC m=+4030.332741937"
Oct 30 19:41:50.257299 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:50.257263    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"]
Oct 30 19:41:50.257752 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:50.257490    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d" podUID="49ca5cb4-d828-417b-841f-c888ee4b5488" containerName="maas-api" containerID="cri-o://2cebd08fa929f7a59cec6c01eeb5f9692fdf1c337ec3ed742bcab8c20e9207a1" gracePeriod=30
Oct 30 19:41:50.388134 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:50.388101    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:50.529610 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:50.529587    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"
Oct 30 19:41:50.622264 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:50.622234    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49ca5cb4-d828-417b-841f-c888ee4b5488-kube-api-access-9z7f7\" (UniqueName: \"kubernetes.io/projected/49ca5cb4-d828-417b-841f-c888ee4b5488-kube-api-access-9z7f7\") pod \"49ca5cb4-d828-417b-841f-c888ee4b5488\" (UID: \"49ca5cb4-d828-417b-841f-c888ee4b5488\") "
Oct 30 19:41:50.622407 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:50.622327    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/49ca5cb4-d828-417b-841f-c888ee4b5488-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49ca5cb4-d828-417b-841f-c888ee4b5488-openshift-service-ca\") pod \"49ca5cb4-d828-417b-841f-c888ee4b5488\" (UID: \"49ca5cb4-d828-417b-841f-c888ee4b5488\") "
Oct 30 19:41:50.622493 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:50.622467    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/49ca5cb4-d828-417b-841f-c888ee4b5488-metrics-certs\" (UniqueName: \"kubernetes.io/secret/49ca5cb4-d828-417b-841f-c888ee4b5488-metrics-certs\") pod \"49ca5cb4-d828-417b-841f-c888ee4b5488\" (UID: \"49ca5cb4-d828-417b-841f-c888ee4b5488\") "
Oct 30 19:41:50.622551 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:50.622505    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/49ca5cb4-d828-417b-841f-c888ee4b5488-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49ca5cb4-d828-417b-841f-c888ee4b5488-maas-api-tls\") pod \"49ca5cb4-d828-417b-841f-c888ee4b5488\" (UID: \"49ca5cb4-d828-417b-841f-c888ee4b5488\") "
Oct 30 19:41:50.622680 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:50.622657    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49ca5cb4-d828-417b-841f-c888ee4b5488-openshift-service-ca" pod "49ca5cb4-d828-417b-841f-c888ee4b5488" (UID: "49ca5cb4-d828-417b-841f-c888ee4b5488"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:41:50.622969 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:50.622926    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49ca5cb4-d828-417b-841f-c888ee4b5488-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:50.624528 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:50.624500    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49ca5cb4-d828-417b-841f-c888ee4b5488-kube-api-access-9z7f7" pod "49ca5cb4-d828-417b-841f-c888ee4b5488" (UID: "49ca5cb4-d828-417b-841f-c888ee4b5488"). InnerVolumeSpecName "kube-api-access-9z7f7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:50.624634 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:50.624616    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49ca5cb4-d828-417b-841f-c888ee4b5488-maas-api-tls" pod "49ca5cb4-d828-417b-841f-c888ee4b5488" (UID: "49ca5cb4-d828-417b-841f-c888ee4b5488"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:41:50.624693 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:50.624676    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49ca5cb4-d828-417b-841f-c888ee4b5488-metrics-certs" pod "49ca5cb4-d828-417b-841f-c888ee4b5488" (UID: "49ca5cb4-d828-417b-841f-c888ee4b5488"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:41:50.723749 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:50.723722    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49ca5cb4-d828-417b-841f-c888ee4b5488-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:50.723749 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:50.723750    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49ca5cb4-d828-417b-841f-c888ee4b5488-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:50.723986 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:50.723763    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9z7f7\" (UniqueName: \"kubernetes.io/projected/49ca5cb4-d828-417b-841f-c888ee4b5488-kube-api-access-9z7f7\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:51.366217 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:51.366186    5820 generic.go:356] "Generic (PLEG): container finished" podID="49ca5cb4-d828-417b-841f-c888ee4b5488" containerID="2cebd08fa929f7a59cec6c01eeb5f9692fdf1c337ec3ed742bcab8c20e9207a1" exitCode=0
Oct 30 19:41:51.366630 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:51.366233    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"
Oct 30 19:41:51.366630 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:51.366269    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d" event={"ID":"49ca5cb4-d828-417b-841f-c888ee4b5488","Type":"ContainerDied","Data":"2cebd08fa929f7a59cec6c01eeb5f9692fdf1c337ec3ed742bcab8c20e9207a1"}
Oct 30 19:41:51.366630 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:51.366320    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d" event={"ID":"49ca5cb4-d828-417b-841f-c888ee4b5488","Type":"ContainerDied","Data":"bb7c24c785e03a6cbc57ce6c4df9d16f44805ef62528bf39bb0fdf66db1c26dd"}
Oct 30 19:41:51.366630 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:51.366345    5820 scope.go:122] "RemoveContainer" containerID="2cebd08fa929f7a59cec6c01eeb5f9692fdf1c337ec3ed742bcab8c20e9207a1"
Oct 30 19:41:51.376114 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:51.376097    5820 scope.go:122] "RemoveContainer" containerID="2cebd08fa929f7a59cec6c01eeb5f9692fdf1c337ec3ed742bcab8c20e9207a1"
Oct 30 19:41:51.376380 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:41:51.376354    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cebd08fa929f7a59cec6c01eeb5f9692fdf1c337ec3ed742bcab8c20e9207a1\": container with ID starting with 2cebd08fa929f7a59cec6c01eeb5f9692fdf1c337ec3ed742bcab8c20e9207a1 not found: ID does not exist" containerID="2cebd08fa929f7a59cec6c01eeb5f9692fdf1c337ec3ed742bcab8c20e9207a1"
Oct 30 19:41:51.376453 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:51.376387    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cebd08fa929f7a59cec6c01eeb5f9692fdf1c337ec3ed742bcab8c20e9207a1"} err="failed to get container status \"2cebd08fa929f7a59cec6c01eeb5f9692fdf1c337ec3ed742bcab8c20e9207a1\": rpc error: code = NotFound desc = could not find container \"2cebd08fa929f7a59cec6c01eeb5f9692fdf1c337ec3ed742bcab8c20e9207a1\": container with ID starting with 2cebd08fa929f7a59cec6c01eeb5f9692fdf1c337ec3ed742bcab8c20e9207a1 not found: ID does not exist"
Oct 30 19:41:51.530404 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:51.530379    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"]
Oct 30 19:41:51.579539 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:51.579506    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-19f6ea-795f5fdc6d-b6p6d"]
Oct 30 19:41:52.047188 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:52.047142    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:52.325771 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:52.325736    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:52.509280 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:52.509247    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:52.985602 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:52.985558    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:53.435595 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:53.435563    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="49ca5cb4-d828-417b-841f-c888ee4b5488" path="/var/lib/kubelet/pods/49ca5cb4-d828-417b-841f-c888ee4b5488/volumes"
Oct 30 19:41:55.476007 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:55.475934    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng"]
Oct 30 19:41:55.882721 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:55.882686    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:56.064174 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:56.064135    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:57.766272 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:57.766237    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:57.866345 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:57.866312    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh"]
Oct 30 19:41:57.977690 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:57.977106    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd"]
Oct 30 19:41:58.276621 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:58.276587    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:58.502107 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:41:58.502076    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:00.091542 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:00.091493    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:00.271588 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:00.271540    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:00.362933 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:00.362857    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw"
Oct 30 19:42:00.363123 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:00.362963    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw"
Oct 30 19:42:00.394117 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:00.394085    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"]
Oct 30 19:42:00.394339 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:00.394315    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9" podUID="c1720171-99ee-4be0-8bbe-f8fdfed25319" containerName="istio-proxy" containerID="cri-o://5bf0b2f26c81b72e3cf5b61d022c597172789cd89c88313676e66b74cccf58ea" gracePeriod=30
Oct 30 19:42:00.448244 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:00.448199    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:00.477314 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:00.477261    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng" podUID="91146e5d-893c-4148-aac2-d128b68567de" containerName="payload-pre-processing" containerID="cri-o://15341471d86e65fda0dc3de3042b79b4eac464ee97cb184a9dd3d1903bf45c28" gracePeriod=25
Oct 30 19:42:01.937386 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:01.937346    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng" podUID="91146e5d-893c-4148-aac2-d128b68567de" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:42:02.837504 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:02.837457    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:02.867176 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:02.867138    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh" podUID="54f97865-2f4e-4aa8-80d8-4e632aaf3847" containerName="payload-processing" containerID="cri-o://92d1ebf2b3a591c4b56f8bf9ca928385413f4db45cd486a845bcc40628c4b8d7" gracePeriod=25
Oct 30 19:42:02.978171 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:02.978124    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd" podUID="4c05e9f8-0218-4456-8f76-1420649f533f" containerName="payload-pre-processing" containerID="cri-o://fd313499defbd0309a13bfaebd633644b3420ffb57cd0b1612bd212a1c9a2308" gracePeriod=25
Oct 30 19:42:04.282926 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.282882    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt" podUID="e31dcc89-d59d-4532-bda4-f28eb3b91095" containerName="main" containerID="cri-o://b18a0bf5f177ef37de8a1bbb51dcfffd0e40743263838f758e3327031c778d08" gracePeriod=45
Oct 30 19:42:04.286647 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.286612    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk" podUID="7300a782-cf82-48ef-a0da-0429b80a3480" containerName="main" containerID="cri-o://34318dabe902981b54edd50a43d1255c77e9bcad416a5d6ea9c2b9a165538923" gracePeriod=45
Oct 30 19:42:04.451748 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.451712    5820 generic.go:356] "Generic (PLEG): container finished" podID="7300a782-cf82-48ef-a0da-0429b80a3480" containerID="34318dabe902981b54edd50a43d1255c77e9bcad416a5d6ea9c2b9a165538923" exitCode=0
Oct 30 19:42:04.451972 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.451768    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk" event={"ID":"7300a782-cf82-48ef-a0da-0429b80a3480","Type":"ContainerDied","Data":"34318dabe902981b54edd50a43d1255c77e9bcad416a5d6ea9c2b9a165538923"}
Oct 30 19:42:04.453639 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.453602    5820 generic.go:356] "Generic (PLEG): container finished" podID="e31dcc89-d59d-4532-bda4-f28eb3b91095" containerID="b18a0bf5f177ef37de8a1bbb51dcfffd0e40743263838f758e3327031c778d08" exitCode=0
Oct 30 19:42:04.453750 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.453674    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt" event={"ID":"e31dcc89-d59d-4532-bda4-f28eb3b91095","Type":"ContainerDied","Data":"b18a0bf5f177ef37de8a1bbb51dcfffd0e40743263838f758e3327031c778d08"}
Oct 30 19:42:04.560606 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.560579    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 19:42:04.563837 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.563818    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 19:42:04.691988 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.691959    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7300a782-cf82-48ef-a0da-0429b80a3480-kube-api-access-mmtd7\" (UniqueName: \"kubernetes.io/projected/7300a782-cf82-48ef-a0da-0429b80a3480-kube-api-access-mmtd7\") pod \"7300a782-cf82-48ef-a0da-0429b80a3480\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") "
Oct 30 19:42:04.692176 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.692010    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-dshm\") pod \"e31dcc89-d59d-4532-bda4-f28eb3b91095\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") "
Oct 30 19:42:04.692176 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.692077    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-tmp-dir\") pod \"7300a782-cf82-48ef-a0da-0429b80a3480\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") "
Oct 30 19:42:04.692176 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.692120    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e31dcc89-d59d-4532-bda4-f28eb3b91095-tls-certs\" (UniqueName: \"kubernetes.io/secret/e31dcc89-d59d-4532-bda4-f28eb3b91095-tls-certs\") pod \"e31dcc89-d59d-4532-bda4-f28eb3b91095\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") "
Oct 30 19:42:04.692176 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.692165    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-tmp-dir\") pod \"e31dcc89-d59d-4532-bda4-f28eb3b91095\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") "
Oct 30 19:42:04.692420 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.692204    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-model-cache\") pod \"7300a782-cf82-48ef-a0da-0429b80a3480\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") "
Oct 30 19:42:04.692420 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.692281    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-dshm\") pod \"7300a782-cf82-48ef-a0da-0429b80a3480\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") "
Oct 30 19:42:04.692420 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.692341    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e31dcc89-d59d-4532-bda4-f28eb3b91095-kube-api-access-wnz87\" (UniqueName: \"kubernetes.io/projected/e31dcc89-d59d-4532-bda4-f28eb3b91095-kube-api-access-wnz87\") pod \"e31dcc89-d59d-4532-bda4-f28eb3b91095\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") "
Oct 30 19:42:04.692420 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.692394    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-home\" (UniqueName: \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-home\") pod \"e31dcc89-d59d-4532-bda4-f28eb3b91095\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") "
Oct 30 19:42:04.692635 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.692436    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-home\" (UniqueName: \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-home\") pod \"7300a782-cf82-48ef-a0da-0429b80a3480\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") "
Oct 30 19:42:04.692635 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.692474    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-model-cache\") pod \"e31dcc89-d59d-4532-bda4-f28eb3b91095\" (UID: \"e31dcc89-d59d-4532-bda4-f28eb3b91095\") "
Oct 30 19:42:04.692635 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.692545    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7300a782-cf82-48ef-a0da-0429b80a3480-tls-certs\" (UniqueName: \"kubernetes.io/secret/7300a782-cf82-48ef-a0da-0429b80a3480-tls-certs\") pod \"7300a782-cf82-48ef-a0da-0429b80a3480\" (UID: \"7300a782-cf82-48ef-a0da-0429b80a3480\") "
Oct 30 19:42:04.692635 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.692541    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-tmp-dir" pod "e31dcc89-d59d-4532-bda4-f28eb3b91095" (UID: "e31dcc89-d59d-4532-bda4-f28eb3b91095"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:04.692928 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.692742    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-home" pod "e31dcc89-d59d-4532-bda4-f28eb3b91095" (UID: "e31dcc89-d59d-4532-bda4-f28eb3b91095"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:04.693021 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.692938    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-model-cache" pod "e31dcc89-d59d-4532-bda4-f28eb3b91095" (UID: "e31dcc89-d59d-4532-bda4-f28eb3b91095"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:04.693425 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.693156    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-model-cache" pod "7300a782-cf82-48ef-a0da-0429b80a3480" (UID: "7300a782-cf82-48ef-a0da-0429b80a3480"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:04.693425 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.693269    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-tmp-dir" pod "7300a782-cf82-48ef-a0da-0429b80a3480" (UID: "7300a782-cf82-48ef-a0da-0429b80a3480"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:04.693633 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.693578    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-home" pod "7300a782-cf82-48ef-a0da-0429b80a3480" (UID: "7300a782-cf82-48ef-a0da-0429b80a3480"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:04.693967 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.693932    5820 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-tmp-dir\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:04.694045 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.693976    5820 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-tmp-dir\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:04.694045 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.693992    5820 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-model-cache\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:04.694045 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.694005    5820 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-home\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:04.694045 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.694018    5820 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-home\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:04.694045 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.694032    5820 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-model-cache\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:04.695198 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.695176    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7300a782-cf82-48ef-a0da-0429b80a3480-kube-api-access-mmtd7" pod "7300a782-cf82-48ef-a0da-0429b80a3480" (UID: "7300a782-cf82-48ef-a0da-0429b80a3480"). InnerVolumeSpecName "kube-api-access-mmtd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:04.695336 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.695307    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-dshm" pod "e31dcc89-d59d-4532-bda4-f28eb3b91095" (UID: "e31dcc89-d59d-4532-bda4-f28eb3b91095"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:04.695620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.695585    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e31dcc89-d59d-4532-bda4-f28eb3b91095-tls-certs" pod "e31dcc89-d59d-4532-bda4-f28eb3b91095" (UID: "e31dcc89-d59d-4532-bda4-f28eb3b91095"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:42:04.695896 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.695874    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-dshm" pod "7300a782-cf82-48ef-a0da-0429b80a3480" (UID: "7300a782-cf82-48ef-a0da-0429b80a3480"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:04.696222 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.696199    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e31dcc89-d59d-4532-bda4-f28eb3b91095-kube-api-access-wnz87" pod "e31dcc89-d59d-4532-bda4-f28eb3b91095" (UID: "e31dcc89-d59d-4532-bda4-f28eb3b91095"). InnerVolumeSpecName "kube-api-access-wnz87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:04.696560 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.696540    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7300a782-cf82-48ef-a0da-0429b80a3480-tls-certs" pod "7300a782-cf82-48ef-a0da-0429b80a3480" (UID: "7300a782-cf82-48ef-a0da-0429b80a3480"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:42:04.794854 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.794689    5820 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7300a782-cf82-48ef-a0da-0429b80a3480-dshm\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:04.794854 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.794732    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnz87\" (UniqueName: \"kubernetes.io/projected/e31dcc89-d59d-4532-bda4-f28eb3b91095-kube-api-access-wnz87\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:04.794854 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.794748    5820 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7300a782-cf82-48ef-a0da-0429b80a3480-tls-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:04.794854 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.794757    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmtd7\" (UniqueName: \"kubernetes.io/projected/7300a782-cf82-48ef-a0da-0429b80a3480-kube-api-access-mmtd7\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:04.794854 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.794772    5820 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e31dcc89-d59d-4532-bda4-f28eb3b91095-dshm\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:04.794854 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:04.794780    5820 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e31dcc89-d59d-4532-bda4-f28eb3b91095-tls-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:05.460527 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.460496    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"
Oct 30 19:42:05.460527 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.460503    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk" event={"ID":"7300a782-cf82-48ef-a0da-0429b80a3480","Type":"ContainerDied","Data":"2e7a4d85199813fc304bae01aa79656c096989d709866c46642ca69f2bd7961d"}
Oct 30 19:42:05.461081 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.460546    5820 scope.go:122] "RemoveContainer" containerID="34318dabe902981b54edd50a43d1255c77e9bcad416a5d6ea9c2b9a165538923"
Oct 30 19:42:05.462101 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.462072    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"
Oct 30 19:42:05.462211 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.462116    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt" event={"ID":"e31dcc89-d59d-4532-bda4-f28eb3b91095","Type":"ContainerDied","Data":"850744d2d248cbf69b5da9c4009286475bea900a37579b32888a38d8b9ab55a0"}
Oct 30 19:42:05.506110 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.506088    5820 scope.go:122] "RemoveContainer" containerID="b18a0bf5f177ef37de8a1bbb51dcfffd0e40743263838f758e3327031c778d08"
Oct 30 19:42:05.626558 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.626524    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"]
Oct 30 19:42:05.628404 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.628379    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-4a7a52/distinct-w2-4a7a52-kserve-867d67787-qdjqk"]
Oct 30 19:42:05.651166 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.651143    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:42:05.747528 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.747435    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"]
Oct 30 19:42:05.753477 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.753451    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-4a7a52/distinct-2-w2-4a7a52-kserve-7bbc6d9956-7gqrt"]
Oct 30 19:42:05.806030 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.805998    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-data\") pod \"c1720171-99ee-4be0-8bbe-f8fdfed25319\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") "
Oct 30 19:42:05.806221 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.806066    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-token\" (UniqueName: \"kubernetes.io/projected/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-token\") pod \"c1720171-99ee-4be0-8bbe-f8fdfed25319\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") "
Oct 30 19:42:05.806221 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.806118    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-envoy\") pod \"c1720171-99ee-4be0-8bbe-f8fdfed25319\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") "
Oct 30 19:42:05.806221 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.806206    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-podinfo\") pod \"c1720171-99ee-4be0-8bbe-f8fdfed25319\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") "
Oct 30 19:42:05.806398 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.806256    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-workload-socket\") pod \"c1720171-99ee-4be0-8bbe-f8fdfed25319\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") "
Oct 30 19:42:05.806398 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.806313    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-workload-certs\") pod \"c1720171-99ee-4be0-8bbe-f8fdfed25319\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") "
Oct 30 19:42:05.806398 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.806312    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-data" pod "c1720171-99ee-4be0-8bbe-f8fdfed25319" (UID: "c1720171-99ee-4be0-8bbe-f8fdfed25319"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:05.806572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.806397    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c1720171-99ee-4be0-8bbe-f8fdfed25319-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1720171-99ee-4be0-8bbe-f8fdfed25319-istiod-ca-cert\") pod \"c1720171-99ee-4be0-8bbe-f8fdfed25319\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") "
Oct 30 19:42:05.806572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.806427    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-credential-socket\") pod \"c1720171-99ee-4be0-8bbe-f8fdfed25319\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") "
Oct 30 19:42:05.806572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.806462    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1720171-99ee-4be0-8bbe-f8fdfed25319-kube-api-access-nmxnt\" (UniqueName: \"kubernetes.io/projected/c1720171-99ee-4be0-8bbe-f8fdfed25319-kube-api-access-nmxnt\") pod \"c1720171-99ee-4be0-8bbe-f8fdfed25319\" (UID: \"c1720171-99ee-4be0-8bbe-f8fdfed25319\") "
Oct 30 19:42:05.806572 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.806551    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-workload-socket" pod "c1720171-99ee-4be0-8bbe-f8fdfed25319" (UID: "c1720171-99ee-4be0-8bbe-f8fdfed25319"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:05.806792 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.806626    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-workload-certs" pod "c1720171-99ee-4be0-8bbe-f8fdfed25319" (UID: "c1720171-99ee-4be0-8bbe-f8fdfed25319"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:05.807075 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.806920    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1720171-99ee-4be0-8bbe-f8fdfed25319-istiod-ca-cert" pod "c1720171-99ee-4be0-8bbe-f8fdfed25319" (UID: "c1720171-99ee-4be0-8bbe-f8fdfed25319"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:42:05.807206 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.807178    5820 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-workload-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:05.807206 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.807208    5820 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1720171-99ee-4be0-8bbe-f8fdfed25319-istiod-ca-cert\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:05.807415 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.807221    5820 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-data\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:05.807415 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.807227    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-credential-socket" pod "c1720171-99ee-4be0-8bbe-f8fdfed25319" (UID: "c1720171-99ee-4be0-8bbe-f8fdfed25319"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:05.807415 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.807238    5820 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-workload-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:05.808533 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.808505    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-token" pod "c1720171-99ee-4be0-8bbe-f8fdfed25319" (UID: "c1720171-99ee-4be0-8bbe-f8fdfed25319"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:05.809016 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.808993    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-podinfo" pod "c1720171-99ee-4be0-8bbe-f8fdfed25319" (UID: "c1720171-99ee-4be0-8bbe-f8fdfed25319"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:42:05.809120 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.808993    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-envoy" pod "c1720171-99ee-4be0-8bbe-f8fdfed25319" (UID: "c1720171-99ee-4be0-8bbe-f8fdfed25319"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:05.809192 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.809177    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1720171-99ee-4be0-8bbe-f8fdfed25319-kube-api-access-nmxnt" pod "c1720171-99ee-4be0-8bbe-f8fdfed25319" (UID: "c1720171-99ee-4be0-8bbe-f8fdfed25319"). InnerVolumeSpecName "kube-api-access-nmxnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:05.908819 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.908790    5820 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-credential-socket\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:05.908819 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.908817    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmxnt\" (UniqueName: \"kubernetes.io/projected/c1720171-99ee-4be0-8bbe-f8fdfed25319-kube-api-access-nmxnt\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:05.908819 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.908826    5820 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-token\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:05.909073 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.908834    5820 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-envoy\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:05.909073 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:05.908843    5820 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1720171-99ee-4be0-8bbe-f8fdfed25319-istio-podinfo\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:06.469826 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:06.469782    5820 generic.go:356] "Generic (PLEG): container finished" podID="c1720171-99ee-4be0-8bbe-f8fdfed25319" containerID="5bf0b2f26c81b72e3cf5b61d022c597172789cd89c88313676e66b74cccf58ea" exitCode=0
Oct 30 19:42:06.470350 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:06.469926    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"
Oct 30 19:42:06.470350 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:06.469928    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9" event={"ID":"c1720171-99ee-4be0-8bbe-f8fdfed25319","Type":"ContainerDied","Data":"5bf0b2f26c81b72e3cf5b61d022c597172789cd89c88313676e66b74cccf58ea"}
Oct 30 19:42:06.470350 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:06.470026    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9" event={"ID":"c1720171-99ee-4be0-8bbe-f8fdfed25319","Type":"ContainerDied","Data":"62373eaa6dc8fda763c1369633eea1fd6b9f06f227cf507d964367ecdd492c2d"}
Oct 30 19:42:06.470350 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:06.470060    5820 scope.go:122] "RemoveContainer" containerID="5bf0b2f26c81b72e3cf5b61d022c597172789cd89c88313676e66b74cccf58ea"
Oct 30 19:42:06.480189 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:06.480170    5820 scope.go:122] "RemoveContainer" containerID="8f6b68d55087a853db1204e07278fe0fa7cba762fed7fd73f9087270a41675d7"
Oct 30 19:42:06.488684 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:06.488669    5820 scope.go:122] "RemoveContainer" containerID="5bf0b2f26c81b72e3cf5b61d022c597172789cd89c88313676e66b74cccf58ea"
Oct 30 19:42:06.488938 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:42:06.488922    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bf0b2f26c81b72e3cf5b61d022c597172789cd89c88313676e66b74cccf58ea\": container with ID starting with 5bf0b2f26c81b72e3cf5b61d022c597172789cd89c88313676e66b74cccf58ea not found: ID does not exist" containerID="5bf0b2f26c81b72e3cf5b61d022c597172789cd89c88313676e66b74cccf58ea"
Oct 30 19:42:06.489007 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:06.488963    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bf0b2f26c81b72e3cf5b61d022c597172789cd89c88313676e66b74cccf58ea"} err="failed to get container status \"5bf0b2f26c81b72e3cf5b61d022c597172789cd89c88313676e66b74cccf58ea\": rpc error: code = NotFound desc = could not find container \"5bf0b2f26c81b72e3cf5b61d022c597172789cd89c88313676e66b74cccf58ea\": container with ID starting with 5bf0b2f26c81b72e3cf5b61d022c597172789cd89c88313676e66b74cccf58ea not found: ID does not exist"
Oct 30 19:42:06.489007 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:06.488983    5820 scope.go:122] "RemoveContainer" containerID="8f6b68d55087a853db1204e07278fe0fa7cba762fed7fd73f9087270a41675d7"
Oct 30 19:42:06.489215 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:42:06.489199    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f6b68d55087a853db1204e07278fe0fa7cba762fed7fd73f9087270a41675d7\": container with ID starting with 8f6b68d55087a853db1204e07278fe0fa7cba762fed7fd73f9087270a41675d7 not found: ID does not exist" containerID="8f6b68d55087a853db1204e07278fe0fa7cba762fed7fd73f9087270a41675d7"
Oct 30 19:42:06.489252 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:06.489218    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f6b68d55087a853db1204e07278fe0fa7cba762fed7fd73f9087270a41675d7"} err="failed to get container status \"8f6b68d55087a853db1204e07278fe0fa7cba762fed7fd73f9087270a41675d7\": rpc error: code = NotFound desc = could not find container \"8f6b68d55087a853db1204e07278fe0fa7cba762fed7fd73f9087270a41675d7\": container with ID starting with 8f6b68d55087a853db1204e07278fe0fa7cba762fed7fd73f9087270a41675d7 not found: ID does not exist"
Oct 30 19:42:06.700510 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:06.700473    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"]
Oct 30 19:42:06.750169 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:06.750088    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-19f6ea-openshift-default-945ccf8c9-lrhc9"]
Oct 30 19:42:07.436653 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:07.436618    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7300a782-cf82-48ef-a0da-0429b80a3480" path="/var/lib/kubelet/pods/7300a782-cf82-48ef-a0da-0429b80a3480/volumes"
Oct 30 19:42:07.437116 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:07.437101    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c1720171-99ee-4be0-8bbe-f8fdfed25319" path="/var/lib/kubelet/pods/c1720171-99ee-4be0-8bbe-f8fdfed25319/volumes"
Oct 30 19:42:07.437494 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:07.437482    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e31dcc89-d59d-4532-bda4-f28eb3b91095" path="/var/lib/kubelet/pods/e31dcc89-d59d-4532-bda4-f28eb3b91095/volumes"
Oct 30 19:42:08.070255 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:08.070217    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:08.203393 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:42:08.203355    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:42:09.598305 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:09.598266    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:10.860903 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:10.860863    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd" podUID="4c05e9f8-0218-4456-8f76-1420649f533f" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:42:10.861324 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:10.860875    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh" podUID="54f97865-2f4e-4aa8-80d8-4e632aaf3847" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:42:11.937162 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:11.937128    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng" podUID="91146e5d-893c-4148-aac2-d128b68567de" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:42:13.092350 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:13.092306    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:13.359940 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:13.359852    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:13.547125 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:13.547076    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:17.878723 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:17.878626    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:20.732547 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:20.732520    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng"
Oct 30 19:42:20.774554 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:20.774480    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91146e5d-893c-4148-aac2-d128b68567de-kube-api-access-n4km9\" (UniqueName: \"kubernetes.io/projected/91146e5d-893c-4148-aac2-d128b68567de-kube-api-access-n4km9\") pod \"91146e5d-893c-4148-aac2-d128b68567de\" (UID: \"91146e5d-893c-4148-aac2-d128b68567de\") "
Oct 30 19:42:20.774709 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:20.774562    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/91146e5d-893c-4148-aac2-d128b68567de-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91146e5d-893c-4148-aac2-d128b68567de-plugins-config-volume\") pod \"91146e5d-893c-4148-aac2-d128b68567de\" (UID: \"91146e5d-893c-4148-aac2-d128b68567de\") "
Oct 30 19:42:20.774938 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:20.774910    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91146e5d-893c-4148-aac2-d128b68567de-plugins-config-volume" pod "91146e5d-893c-4148-aac2-d128b68567de" (UID: "91146e5d-893c-4148-aac2-d128b68567de"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:42:20.775074 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:20.775035    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91146e5d-893c-4148-aac2-d128b68567de-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:20.777076 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:20.777053    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91146e5d-893c-4148-aac2-d128b68567de-kube-api-access-n4km9" pod "91146e5d-893c-4148-aac2-d128b68567de" (UID: "91146e5d-893c-4148-aac2-d128b68567de"). InnerVolumeSpecName "kube-api-access-n4km9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:20.861630 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:20.861594    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd" podUID="4c05e9f8-0218-4456-8f76-1420649f533f" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:42:20.861814 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:20.861694    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh" podUID="54f97865-2f4e-4aa8-80d8-4e632aaf3847" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:42:20.876600 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:20.876562    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4km9\" (UniqueName: \"kubernetes.io/projected/91146e5d-893c-4148-aac2-d128b68567de-kube-api-access-n4km9\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:21.572450 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:21.572410    5820 generic.go:356] "Generic (PLEG): container finished" podID="91146e5d-893c-4148-aac2-d128b68567de" containerID="15341471d86e65fda0dc3de3042b79b4eac464ee97cb184a9dd3d1903bf45c28" exitCode=0
Oct 30 19:42:21.572618 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:21.572459    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng"
Oct 30 19:42:21.572618 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:21.572495    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng" event={"ID":"91146e5d-893c-4148-aac2-d128b68567de","Type":"ContainerDied","Data":"15341471d86e65fda0dc3de3042b79b4eac464ee97cb184a9dd3d1903bf45c28"}
Oct 30 19:42:21.572618 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:21.572555    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng" event={"ID":"91146e5d-893c-4148-aac2-d128b68567de","Type":"ContainerDied","Data":"629a556e840336f9daf4caf8abd78f977fe41a3852e4c7ddd90715b8499365a0"}
Oct 30 19:42:21.572618 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:21.572577    5820 scope.go:122] "RemoveContainer" containerID="15341471d86e65fda0dc3de3042b79b4eac464ee97cb184a9dd3d1903bf45c28"
Oct 30 19:42:21.581677 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:21.581645    5820 scope.go:122] "RemoveContainer" containerID="15341471d86e65fda0dc3de3042b79b4eac464ee97cb184a9dd3d1903bf45c28"
Oct 30 19:42:21.581928 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:42:21.581912    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15341471d86e65fda0dc3de3042b79b4eac464ee97cb184a9dd3d1903bf45c28\": container with ID starting with 15341471d86e65fda0dc3de3042b79b4eac464ee97cb184a9dd3d1903bf45c28 not found: ID does not exist" containerID="15341471d86e65fda0dc3de3042b79b4eac464ee97cb184a9dd3d1903bf45c28"
Oct 30 19:42:21.582013 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:21.581934    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15341471d86e65fda0dc3de3042b79b4eac464ee97cb184a9dd3d1903bf45c28"} err="failed to get container status \"15341471d86e65fda0dc3de3042b79b4eac464ee97cb184a9dd3d1903bf45c28\": rpc error: code = NotFound desc = could not find container \"15341471d86e65fda0dc3de3042b79b4eac464ee97cb184a9dd3d1903bf45c28\": container with ID starting with 15341471d86e65fda0dc3de3042b79b4eac464ee97cb184a9dd3d1903bf45c28 not found: ID does not exist"
Oct 30 19:42:21.736352 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:21.736271    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng"]
Oct 30 19:42:21.785761 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:21.785728    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-4a7a52-f65d8db55-c6mng"]
Oct 30 19:42:21.920933 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:21.920899    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"]
Oct 30 19:42:21.921149 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:21.921123    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg" podUID="9e354da3-bcc8-41f9-ad39-ffdf8b9596c6" containerName="maas-api" containerID="cri-o://10ca758782f0f624b096ba8fc540107952e5ad9ad4f2e731da13c66be6841c41" gracePeriod=30
Oct 30 19:42:22.063163 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.063123    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:22.178227 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.178204    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"
Oct 30 19:42:22.292462 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.292331    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-kube-api-access-9pl4j\" (UniqueName: \"kubernetes.io/projected/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-kube-api-access-9pl4j\") pod \"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6\" (UID: \"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6\") "
Oct 30 19:42:22.292462 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.292436    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-metrics-certs\") pod \"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6\" (UID: \"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6\") "
Oct 30 19:42:22.292738 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.292550    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-maas-api-tls\") pod \"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6\" (UID: \"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6\") "
Oct 30 19:42:22.292738 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.292656    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-openshift-service-ca\") pod \"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6\" (UID: \"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6\") "
Oct 30 19:42:22.293345 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.293311    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-openshift-service-ca" pod "9e354da3-bcc8-41f9-ad39-ffdf8b9596c6" (UID: "9e354da3-bcc8-41f9-ad39-ffdf8b9596c6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:42:22.294962 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.294914    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-kube-api-access-9pl4j" pod "9e354da3-bcc8-41f9-ad39-ffdf8b9596c6" (UID: "9e354da3-bcc8-41f9-ad39-ffdf8b9596c6"). InnerVolumeSpecName "kube-api-access-9pl4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:22.295219 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.295199    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-metrics-certs" pod "9e354da3-bcc8-41f9-ad39-ffdf8b9596c6" (UID: "9e354da3-bcc8-41f9-ad39-ffdf8b9596c6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:42:22.295381 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.295359    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-maas-api-tls" pod "9e354da3-bcc8-41f9-ad39-ffdf8b9596c6" (UID: "9e354da3-bcc8-41f9-ad39-ffdf8b9596c6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:42:22.394120 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.394076    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pl4j\" (UniqueName: \"kubernetes.io/projected/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-kube-api-access-9pl4j\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:22.394120 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.394116    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:22.394120 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.394125    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:22.394120 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.394134    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:22.580090 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.580051    5820 generic.go:356] "Generic (PLEG): container finished" podID="9e354da3-bcc8-41f9-ad39-ffdf8b9596c6" containerID="10ca758782f0f624b096ba8fc540107952e5ad9ad4f2e731da13c66be6841c41" exitCode=0
Oct 30 19:42:22.580270 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.580103    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"
Oct 30 19:42:22.580270 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.580109    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg" event={"ID":"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6","Type":"ContainerDied","Data":"10ca758782f0f624b096ba8fc540107952e5ad9ad4f2e731da13c66be6841c41"}
Oct 30 19:42:22.580270 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.580161    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg" event={"ID":"9e354da3-bcc8-41f9-ad39-ffdf8b9596c6","Type":"ContainerDied","Data":"319303f4e19877510dc23a2559c4881b370c88b4e32ecf7668f2cdf9636fa191"}
Oct 30 19:42:22.580270 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.580192    5820 scope.go:122] "RemoveContainer" containerID="10ca758782f0f624b096ba8fc540107952e5ad9ad4f2e731da13c66be6841c41"
Oct 30 19:42:22.589803 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.589782    5820 scope.go:122] "RemoveContainer" containerID="10ca758782f0f624b096ba8fc540107952e5ad9ad4f2e731da13c66be6841c41"
Oct 30 19:42:22.590095 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:42:22.590076    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10ca758782f0f624b096ba8fc540107952e5ad9ad4f2e731da13c66be6841c41\": container with ID starting with 10ca758782f0f624b096ba8fc540107952e5ad9ad4f2e731da13c66be6841c41 not found: ID does not exist" containerID="10ca758782f0f624b096ba8fc540107952e5ad9ad4f2e731da13c66be6841c41"
Oct 30 19:42:22.590154 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.590105    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10ca758782f0f624b096ba8fc540107952e5ad9ad4f2e731da13c66be6841c41"} err="failed to get container status \"10ca758782f0f624b096ba8fc540107952e5ad9ad4f2e731da13c66be6841c41\": rpc error: code = NotFound desc = could not find container \"10ca758782f0f624b096ba8fc540107952e5ad9ad4f2e731da13c66be6841c41\": container with ID starting with 10ca758782f0f624b096ba8fc540107952e5ad9ad4f2e731da13c66be6841c41 not found: ID does not exist"
Oct 30 19:42:22.745652 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.745619    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"]
Oct 30 19:42:22.795147 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:22.795115    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-d4f84d-7699db874b-876zg"]
Oct 30 19:42:23.126511 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.126447    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh"
Oct 30 19:42:23.203303 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.203273    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54f97865-2f4e-4aa8-80d8-4e632aaf3847-kube-api-access-zt5j5\" (UniqueName: \"kubernetes.io/projected/54f97865-2f4e-4aa8-80d8-4e632aaf3847-kube-api-access-zt5j5\") pod \"54f97865-2f4e-4aa8-80d8-4e632aaf3847\" (UID: \"54f97865-2f4e-4aa8-80d8-4e632aaf3847\") "
Oct 30 19:42:23.203491 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.203423    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/54f97865-2f4e-4aa8-80d8-4e632aaf3847-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54f97865-2f4e-4aa8-80d8-4e632aaf3847-plugins-config-volume\") pod \"54f97865-2f4e-4aa8-80d8-4e632aaf3847\" (UID: \"54f97865-2f4e-4aa8-80d8-4e632aaf3847\") "
Oct 30 19:42:23.203787 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.203759    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54f97865-2f4e-4aa8-80d8-4e632aaf3847-plugins-config-volume" pod "54f97865-2f4e-4aa8-80d8-4e632aaf3847" (UID: "54f97865-2f4e-4aa8-80d8-4e632aaf3847"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:42:23.205567 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.205545    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54f97865-2f4e-4aa8-80d8-4e632aaf3847-kube-api-access-zt5j5" pod "54f97865-2f4e-4aa8-80d8-4e632aaf3847" (UID: "54f97865-2f4e-4aa8-80d8-4e632aaf3847"). InnerVolumeSpecName "kube-api-access-zt5j5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:23.223358 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.223335    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd"
Oct 30 19:42:23.304311 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.304266    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c05e9f8-0218-4456-8f76-1420649f533f-kube-api-access-snkrl\" (UniqueName: \"kubernetes.io/projected/4c05e9f8-0218-4456-8f76-1420649f533f-kube-api-access-snkrl\") pod \"4c05e9f8-0218-4456-8f76-1420649f533f\" (UID: \"4c05e9f8-0218-4456-8f76-1420649f533f\") "
Oct 30 19:42:23.304531 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.304444    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4c05e9f8-0218-4456-8f76-1420649f533f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c05e9f8-0218-4456-8f76-1420649f533f-plugins-config-volume\") pod \"4c05e9f8-0218-4456-8f76-1420649f533f\" (UID: \"4c05e9f8-0218-4456-8f76-1420649f533f\") "
Oct 30 19:42:23.304772 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.304754    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54f97865-2f4e-4aa8-80d8-4e632aaf3847-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:23.304845 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.304782    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zt5j5\" (UniqueName: \"kubernetes.io/projected/54f97865-2f4e-4aa8-80d8-4e632aaf3847-kube-api-access-zt5j5\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:23.304845 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.304756    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c05e9f8-0218-4456-8f76-1420649f533f-plugins-config-volume" pod "4c05e9f8-0218-4456-8f76-1420649f533f" (UID: "4c05e9f8-0218-4456-8f76-1420649f533f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:42:23.306596 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.306570    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c05e9f8-0218-4456-8f76-1420649f533f-kube-api-access-snkrl" pod "4c05e9f8-0218-4456-8f76-1420649f533f" (UID: "4c05e9f8-0218-4456-8f76-1420649f533f"). InnerVolumeSpecName "kube-api-access-snkrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:23.405818 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.405732    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snkrl\" (UniqueName: \"kubernetes.io/projected/4c05e9f8-0218-4456-8f76-1420649f533f-kube-api-access-snkrl\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:23.405818 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.405768    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c05e9f8-0218-4456-8f76-1420649f533f-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:23.436225 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.436193    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="91146e5d-893c-4148-aac2-d128b68567de" path="/var/lib/kubelet/pods/91146e5d-893c-4148-aac2-d128b68567de/volumes"
Oct 30 19:42:23.436629 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.436616    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9e354da3-bcc8-41f9-ad39-ffdf8b9596c6" path="/var/lib/kubelet/pods/9e354da3-bcc8-41f9-ad39-ffdf8b9596c6/volumes"
Oct 30 19:42:23.540979 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.540933    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"]
Oct 30 19:42:23.541438 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541426    5820 cpu_manager.go:414] "containerMap: removing container" podUID="7300a782-cf82-48ef-a0da-0429b80a3480" containerName="main"
Oct 30 19:42:23.541480 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541440    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7300a782-cf82-48ef-a0da-0429b80a3480" containerName="main"
Oct 30 19:42:23.541480 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541451    5820 cpu_manager.go:414] "containerMap: removing container" podUID="c1720171-99ee-4be0-8bbe-f8fdfed25319" containerName="istio-proxy"
Oct 30 19:42:23.541480 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541456    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c1720171-99ee-4be0-8bbe-f8fdfed25319" containerName="istio-proxy"
Oct 30 19:42:23.541480 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541463    5820 cpu_manager.go:414] "containerMap: removing container" podUID="54f97865-2f4e-4aa8-80d8-4e632aaf3847" containerName="payload-processing"
Oct 30 19:42:23.541480 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541468    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54f97865-2f4e-4aa8-80d8-4e632aaf3847" containerName="payload-processing"
Oct 30 19:42:23.541480 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541476    5820 cpu_manager.go:414] "containerMap: removing container" podUID="9e354da3-bcc8-41f9-ad39-ffdf8b9596c6" containerName="maas-api"
Oct 30 19:42:23.541480 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541481    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e354da3-bcc8-41f9-ad39-ffdf8b9596c6" containerName="maas-api"
Oct 30 19:42:23.541711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541496    5820 cpu_manager.go:414] "containerMap: removing container" podUID="f4415a3e-5238-48e7-a6b2-e97c1904381f" containerName="istio-proxy"
Oct 30 19:42:23.541711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541501    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4415a3e-5238-48e7-a6b2-e97c1904381f" containerName="istio-proxy"
Oct 30 19:42:23.541711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541513    5820 cpu_manager.go:414] "containerMap: removing container" podUID="91146e5d-893c-4148-aac2-d128b68567de" containerName="payload-pre-processing"
Oct 30 19:42:23.541711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541519    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91146e5d-893c-4148-aac2-d128b68567de" containerName="payload-pre-processing"
Oct 30 19:42:23.541711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541527    5820 cpu_manager.go:414] "containerMap: removing container" podUID="4c05e9f8-0218-4456-8f76-1420649f533f" containerName="payload-pre-processing"
Oct 30 19:42:23.541711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541531    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c05e9f8-0218-4456-8f76-1420649f533f" containerName="payload-pre-processing"
Oct 30 19:42:23.541711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541546    5820 cpu_manager.go:414] "containerMap: removing container" podUID="e31dcc89-d59d-4532-bda4-f28eb3b91095" containerName="main"
Oct 30 19:42:23.541711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541550    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e31dcc89-d59d-4532-bda4-f28eb3b91095" containerName="main"
Oct 30 19:42:23.541711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541558    5820 cpu_manager.go:414] "containerMap: removing container" podUID="49ca5cb4-d828-417b-841f-c888ee4b5488" containerName="maas-api"
Oct 30 19:42:23.541711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541562    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49ca5cb4-d828-417b-841f-c888ee4b5488" containerName="maas-api"
Oct 30 19:42:23.541711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541628    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4415a3e-5238-48e7-a6b2-e97c1904381f" containerName="istio-proxy"
Oct 30 19:42:23.541711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541637    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="91146e5d-893c-4148-aac2-d128b68567de" containerName="payload-pre-processing"
Oct 30 19:42:23.541711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541645    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c05e9f8-0218-4456-8f76-1420649f533f" containerName="payload-pre-processing"
Oct 30 19:42:23.541711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541652    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="c1720171-99ee-4be0-8bbe-f8fdfed25319" containerName="istio-proxy"
Oct 30 19:42:23.541711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541657    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="e31dcc89-d59d-4532-bda4-f28eb3b91095" containerName="main"
Oct 30 19:42:23.541711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541664    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4415a3e-5238-48e7-a6b2-e97c1904381f" containerName="istio-proxy"
Oct 30 19:42:23.541711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541670    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="49ca5cb4-d828-417b-841f-c888ee4b5488" containerName="maas-api"
Oct 30 19:42:23.541711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541678    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="7300a782-cf82-48ef-a0da-0429b80a3480" containerName="main"
Oct 30 19:42:23.541711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541683    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="c1720171-99ee-4be0-8bbe-f8fdfed25319" containerName="istio-proxy"
Oct 30 19:42:23.541711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541689    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="54f97865-2f4e-4aa8-80d8-4e632aaf3847" containerName="payload-processing"
Oct 30 19:42:23.541711 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.541694    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e354da3-bcc8-41f9-ad39-ffdf8b9596c6" containerName="maas-api"
Oct 30 19:42:23.546072 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.546053    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"
Oct 30 19:42:23.574924 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.574890    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:23.586676 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.586651    5820 generic.go:356] "Generic (PLEG): container finished" podID="4c05e9f8-0218-4456-8f76-1420649f533f" containerID="fd313499defbd0309a13bfaebd633644b3420ffb57cd0b1612bd212a1c9a2308" exitCode=0
Oct 30 19:42:23.586793 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.586694    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd"
Oct 30 19:42:23.586793 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.586729    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd" event={"ID":"4c05e9f8-0218-4456-8f76-1420649f533f","Type":"ContainerDied","Data":"fd313499defbd0309a13bfaebd633644b3420ffb57cd0b1612bd212a1c9a2308"}
Oct 30 19:42:23.586793 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.586778    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd" event={"ID":"4c05e9f8-0218-4456-8f76-1420649f533f","Type":"ContainerDied","Data":"f6d205836c756e81c0bf1b42b32f077e60ce38aafad7cf54b716ed8fc562b213"}
Oct 30 19:42:23.586976 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.586801    5820 scope.go:122] "RemoveContainer" containerID="fd313499defbd0309a13bfaebd633644b3420ffb57cd0b1612bd212a1c9a2308"
Oct 30 19:42:23.588255 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.588226    5820 generic.go:356] "Generic (PLEG): container finished" podID="54f97865-2f4e-4aa8-80d8-4e632aaf3847" containerID="92d1ebf2b3a591c4b56f8bf9ca928385413f4db45cd486a845bcc40628c4b8d7" exitCode=0
Oct 30 19:42:23.588386 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.588279    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh"
Oct 30 19:42:23.588386 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.588315    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh" event={"ID":"54f97865-2f4e-4aa8-80d8-4e632aaf3847","Type":"ContainerDied","Data":"92d1ebf2b3a591c4b56f8bf9ca928385413f4db45cd486a845bcc40628c4b8d7"}
Oct 30 19:42:23.588386 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.588377    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh" event={"ID":"54f97865-2f4e-4aa8-80d8-4e632aaf3847","Type":"ContainerDied","Data":"7894a9adbbbeebb7d6de34a85949cba243e091d9dc4a5d96f5424dcb4ac09faa"}
Oct 30 19:42:23.597350 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.597330    5820 scope.go:122] "RemoveContainer" containerID="fd313499defbd0309a13bfaebd633644b3420ffb57cd0b1612bd212a1c9a2308"
Oct 30 19:42:23.597624 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:42:23.597602    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd313499defbd0309a13bfaebd633644b3420ffb57cd0b1612bd212a1c9a2308\": container with ID starting with fd313499defbd0309a13bfaebd633644b3420ffb57cd0b1612bd212a1c9a2308 not found: ID does not exist" containerID="fd313499defbd0309a13bfaebd633644b3420ffb57cd0b1612bd212a1c9a2308"
Oct 30 19:42:23.597706 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.597634    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd313499defbd0309a13bfaebd633644b3420ffb57cd0b1612bd212a1c9a2308"} err="failed to get container status \"fd313499defbd0309a13bfaebd633644b3420ffb57cd0b1612bd212a1c9a2308\": rpc error: code = NotFound desc = could not find container \"fd313499defbd0309a13bfaebd633644b3420ffb57cd0b1612bd212a1c9a2308\": container with ID starting with fd313499defbd0309a13bfaebd633644b3420ffb57cd0b1612bd212a1c9a2308 not found: ID does not exist"
Oct 30 19:42:23.597706 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.597652    5820 scope.go:122] "RemoveContainer" containerID="92d1ebf2b3a591c4b56f8bf9ca928385413f4db45cd486a845bcc40628c4b8d7"
Oct 30 19:42:23.598158 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.598138    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-5da1ae-serving-cert\""
Oct 30 19:42:23.607594 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.607559    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"]
Oct 30 19:42:23.608202 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.608181    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bb27649-8419-414f-a8f0-23a1838ea701-maas-api-tls\") pod \"maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck\" (UID: \"3bb27649-8419-414f-a8f0-23a1838ea701\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"
Oct 30 19:42:23.608314 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.608222    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bb27649-8419-414f-a8f0-23a1838ea701-metrics-certs\") pod \"maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck\" (UID: \"3bb27649-8419-414f-a8f0-23a1838ea701\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"
Oct 30 19:42:23.608453 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.608426    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bb27649-8419-414f-a8f0-23a1838ea701-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck\" (UID: \"3bb27649-8419-414f-a8f0-23a1838ea701\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"
Oct 30 19:42:23.608527 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.608503    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmwfv\" (UniqueName: \"kubernetes.io/projected/3bb27649-8419-414f-a8f0-23a1838ea701-kube-api-access-hmwfv\") pod \"maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck\" (UID: \"3bb27649-8419-414f-a8f0-23a1838ea701\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"
Oct 30 19:42:23.611430 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.611410    5820 scope.go:122] "RemoveContainer" containerID="92d1ebf2b3a591c4b56f8bf9ca928385413f4db45cd486a845bcc40628c4b8d7"
Oct 30 19:42:23.611758 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:42:23.611736    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92d1ebf2b3a591c4b56f8bf9ca928385413f4db45cd486a845bcc40628c4b8d7\": container with ID starting with 92d1ebf2b3a591c4b56f8bf9ca928385413f4db45cd486a845bcc40628c4b8d7 not found: ID does not exist" containerID="92d1ebf2b3a591c4b56f8bf9ca928385413f4db45cd486a845bcc40628c4b8d7"
Oct 30 19:42:23.611808 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.611767    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92d1ebf2b3a591c4b56f8bf9ca928385413f4db45cd486a845bcc40628c4b8d7"} err="failed to get container status \"92d1ebf2b3a591c4b56f8bf9ca928385413f4db45cd486a845bcc40628c4b8d7\": rpc error: code = NotFound desc = could not find container \"92d1ebf2b3a591c4b56f8bf9ca928385413f4db45cd486a845bcc40628c4b8d7\": container with ID starting with 92d1ebf2b3a591c4b56f8bf9ca928385413f4db45cd486a845bcc40628c4b8d7 not found: ID does not exist"
Oct 30 19:42:23.709163 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.709086    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmwfv\" (UniqueName: \"kubernetes.io/projected/3bb27649-8419-414f-a8f0-23a1838ea701-kube-api-access-hmwfv\") pod \"maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck\" (UID: \"3bb27649-8419-414f-a8f0-23a1838ea701\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"
Oct 30 19:42:23.709163 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.709132    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bb27649-8419-414f-a8f0-23a1838ea701-maas-api-tls\") pod \"maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck\" (UID: \"3bb27649-8419-414f-a8f0-23a1838ea701\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"
Oct 30 19:42:23.709367 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.709169    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bb27649-8419-414f-a8f0-23a1838ea701-metrics-certs\") pod \"maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck\" (UID: \"3bb27649-8419-414f-a8f0-23a1838ea701\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"
Oct 30 19:42:23.709367 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.709239    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bb27649-8419-414f-a8f0-23a1838ea701-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck\" (UID: \"3bb27649-8419-414f-a8f0-23a1838ea701\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"
Oct 30 19:42:23.709441 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:42:23.709391    5820 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-serving-cert: secret "maas-api-e2e-extmod-b-5da1ae-serving-cert" not found
Oct 30 19:42:23.709477 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:42:23.709459    5820 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bb27649-8419-414f-a8f0-23a1838ea701-maas-api-tls podName:3bb27649-8419-414f-a8f0-23a1838ea701 nodeName:}" failed. No retries permitted until 2026-10-30 19:42:24.209437553 +0000 UTC m=+4064.921186243 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3bb27649-8419-414f-a8f0-23a1838ea701-maas-api-tls") pod "maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck" (UID: "3bb27649-8419-414f-a8f0-23a1838ea701") : secret "maas-api-e2e-extmod-b-5da1ae-serving-cert" not found
Oct 30 19:42:23.709990 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.709968    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bb27649-8419-414f-a8f0-23a1838ea701-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck\" (UID: \"3bb27649-8419-414f-a8f0-23a1838ea701\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"
Oct 30 19:42:23.711702 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.711682    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bb27649-8419-414f-a8f0-23a1838ea701-metrics-certs\") pod \"maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck\" (UID: \"3bb27649-8419-414f-a8f0-23a1838ea701\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"
Oct 30 19:42:23.773177 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.772967    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmwfv\" (UniqueName: \"kubernetes.io/projected/3bb27649-8419-414f-a8f0-23a1838ea701-kube-api-access-hmwfv\") pod \"maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck\" (UID: \"3bb27649-8419-414f-a8f0-23a1838ea701\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"
Oct 30 19:42:23.774877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.774847    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:23.775257 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.775240    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd"]
Oct 30 19:42:23.781050 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.781030    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d4f84d-5dddc8f7bc-jt2xd"]
Oct 30 19:42:23.888561 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.888522    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh"]
Oct 30 19:42:23.893446 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.893412    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-d4f84d-787bf5c664-vnswh"]
Oct 30 19:42:23.955413 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:23.954085    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:24.215285 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:24.215246    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bb27649-8419-414f-a8f0-23a1838ea701-maas-api-tls\") pod \"maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck\" (UID: \"3bb27649-8419-414f-a8f0-23a1838ea701\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"
Oct 30 19:42:24.217762 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:24.217743    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bb27649-8419-414f-a8f0-23a1838ea701-maas-api-tls\") pod \"maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck\" (UID: \"3bb27649-8419-414f-a8f0-23a1838ea701\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"
Oct 30 19:42:24.459336 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:24.459301    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"
Oct 30 19:42:24.565312 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:24.565284    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:24.833912 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:24.833889    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"]
Oct 30 19:42:24.836300 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:42:24.836269    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bb27649_8419_414f_a8f0_23a1838ea701.slice/crio-6f9d039d4f36679241bef77f6056a403a0a173d3d0f7214fc98ddf0ced95492b WatchSource:0}: Error finding container 6f9d039d4f36679241bef77f6056a403a0a173d3d0f7214fc98ddf0ced95492b: Status 404 returned error can't find the container with id 6f9d039d4f36679241bef77f6056a403a0a173d3d0f7214fc98ddf0ced95492b
Oct 30 19:42:24.837699 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:24.837680    5820 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:42:25.436111 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:25.436078    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4c05e9f8-0218-4456-8f76-1420649f533f" path="/var/lib/kubelet/pods/4c05e9f8-0218-4456-8f76-1420649f533f/volumes"
Oct 30 19:42:25.436453 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:25.436440    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="54f97865-2f4e-4aa8-80d8-4e632aaf3847" path="/var/lib/kubelet/pods/54f97865-2f4e-4aa8-80d8-4e632aaf3847/volumes"
Oct 30 19:42:25.603129 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:25.603092    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck" event={"ID":"3bb27649-8419-414f-a8f0-23a1838ea701","Type":"ContainerStarted","Data":"6f9d039d4f36679241bef77f6056a403a0a173d3d0f7214fc98ddf0ced95492b"}
Oct 30 19:42:26.609819 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:26.609778    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck" event={"ID":"3bb27649-8419-414f-a8f0-23a1838ea701","Type":"ContainerStarted","Data":"e276416a9b136f00bec4b107ecfb9ea9e2830da3c8267bf2b28e4404f488e5b8"}
Oct 30 19:42:26.610289 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:26.609857    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"
Oct 30 19:42:26.677623 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:26.677587    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:26.728793 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:26.728741    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck" podStartSLOduration=2.818816649 podStartE2EDuration="3.728727366s" totalImagesPullingTime="909.910717ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:42:26.726611669 +0000 UTC m=+4067.438360367" watchObservedRunningTime="2026-10-30 19:42:26.728727366 +0000 UTC m=+4067.440476086"
Oct 30 19:42:27.363692 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:27.363658    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:27.545329 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:27.545287    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:27.915677 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:27.915641    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:29.353703 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:29.353668    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:31.891806 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:31.891767    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:32.127580 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:32.127540    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:32.316990 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:32.316933    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:32.620751 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:32.620667    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"
Oct 30 19:42:32.932252 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:32.932156    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:33.844347 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:33.844302    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:35.696393 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:35.696341    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:35.965970 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:35.965860    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:36.542300 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:36.542244    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:37.809539 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:37.809501    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"]
Oct 30 19:42:38.041706 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:38.041657    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:38.203730 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:42:38.203641    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:42:40.902048 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:40.901922    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:43.930555 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:43.930518    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:44.163524 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:44.163484    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:44.423697 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:44.423635    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:44.875147 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:44.875104    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:44.996898 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:44.996860    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx"]
Oct 30 19:42:45.123455 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:45.123423    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw"]
Oct 30 19:42:45.423829 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:45.423792    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:45.605785 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:45.605749    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:46.300622 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:46.300591    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:49.930778 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:49.930732    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:49.997232 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:49.997168    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx" podUID="b576ab89-f39f-49d9-964b-333d324a9c3d" containerName="payload-processing" containerID="cri-o://7f959ec52dc1a86d1c9688db7f0b34c5af5015db283923081a456259c3445368" gracePeriod=25
Oct 30 19:42:50.123764 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:50.123721    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw" podUID="c01e0998-f450-41a4-be3f-c9a94ea6ff59" containerName="payload-pre-processing" containerID="cri-o://9f1993aab2fff61086fab35dd9129d09178456f55764b94d5753e692642bd072" gracePeriod=25
Oct 30 19:42:51.775219 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:51.775179    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:52.830864 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:52.830821    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n" podUID="80773b82-666e-428b-aeec-361a6c9cd6ac" containerName="main" containerID="cri-o://12f9d851304b567522173c6e48d033d723d953a0a977bea14c94466e80c93d2e" gracePeriod=45
Oct 30 19:42:53.085823 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.085771    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 19:42:53.210598 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.210539    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-model-cache\") pod \"80773b82-666e-428b-aeec-361a6c9cd6ac\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") "
Oct 30 19:42:53.210834 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.210601    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-dshm\" (UniqueName: \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-dshm\") pod \"80773b82-666e-428b-aeec-361a6c9cd6ac\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") "
Oct 30 19:42:53.210834 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.210725    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/80773b82-666e-428b-aeec-361a6c9cd6ac-tls-certs\" (UniqueName: \"kubernetes.io/secret/80773b82-666e-428b-aeec-361a6c9cd6ac-tls-certs\") pod \"80773b82-666e-428b-aeec-361a6c9cd6ac\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") "
Oct 30 19:42:53.210834 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.210761    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/80773b82-666e-428b-aeec-361a6c9cd6ac-kube-api-access-l5rrk\" (UniqueName: \"kubernetes.io/projected/80773b82-666e-428b-aeec-361a6c9cd6ac-kube-api-access-l5rrk\") pod \"80773b82-666e-428b-aeec-361a6c9cd6ac\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") "
Oct 30 19:42:53.210834 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.210796    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-tmp-dir\") pod \"80773b82-666e-428b-aeec-361a6c9cd6ac\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") "
Oct 30 19:42:53.211141 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.210860    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-home\" (UniqueName: \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-home\") pod \"80773b82-666e-428b-aeec-361a6c9cd6ac\" (UID: \"80773b82-666e-428b-aeec-361a6c9cd6ac\") "
Oct 30 19:42:53.211523 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.211493    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-model-cache" pod "80773b82-666e-428b-aeec-361a6c9cd6ac" (UID: "80773b82-666e-428b-aeec-361a6c9cd6ac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:53.211627 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.211533    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-home" pod "80773b82-666e-428b-aeec-361a6c9cd6ac" (UID: "80773b82-666e-428b-aeec-361a6c9cd6ac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:53.211712 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.211689    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-tmp-dir" pod "80773b82-666e-428b-aeec-361a6c9cd6ac" (UID: "80773b82-666e-428b-aeec-361a6c9cd6ac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:53.214505 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.214470    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80773b82-666e-428b-aeec-361a6c9cd6ac-kube-api-access-l5rrk" pod "80773b82-666e-428b-aeec-361a6c9cd6ac" (UID: "80773b82-666e-428b-aeec-361a6c9cd6ac"). InnerVolumeSpecName "kube-api-access-l5rrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:53.214772 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.214739    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80773b82-666e-428b-aeec-361a6c9cd6ac-tls-certs" pod "80773b82-666e-428b-aeec-361a6c9cd6ac" (UID: "80773b82-666e-428b-aeec-361a6c9cd6ac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:42:53.215065 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.215034    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-dshm" pod "80773b82-666e-428b-aeec-361a6c9cd6ac" (UID: "80773b82-666e-428b-aeec-361a6c9cd6ac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:53.312638 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.312605    5820 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-home\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:53.312638 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.312637    5820 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-model-cache\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:53.312638 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.312647    5820 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-dshm\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:53.312889 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.312655    5820 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80773b82-666e-428b-aeec-361a6c9cd6ac-tls-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:53.312889 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.312665    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5rrk\" (UniqueName: \"kubernetes.io/projected/80773b82-666e-428b-aeec-361a6c9cd6ac-kube-api-access-l5rrk\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:53.312889 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.312697    5820 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80773b82-666e-428b-aeec-361a6c9cd6ac-tmp-dir\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:53.752735 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.752651    5820 generic.go:356] "Generic (PLEG): container finished" podID="80773b82-666e-428b-aeec-361a6c9cd6ac" containerID="12f9d851304b567522173c6e48d033d723d953a0a977bea14c94466e80c93d2e" exitCode=0
Oct 30 19:42:53.752735 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.752702    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n" event={"ID":"80773b82-666e-428b-aeec-361a6c9cd6ac","Type":"ContainerDied","Data":"12f9d851304b567522173c6e48d033d723d953a0a977bea14c94466e80c93d2e"}
Oct 30 19:42:53.752735 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.752708    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"
Oct 30 19:42:53.753043 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.752741    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n" event={"ID":"80773b82-666e-428b-aeec-361a6c9cd6ac","Type":"ContainerDied","Data":"9038a18ca663dba7b37f6c6055afa79657d36eba2df6ec2831f36bb0be4c0307"}
Oct 30 19:42:53.753043 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.752761    5820 scope.go:122] "RemoveContainer" containerID="12f9d851304b567522173c6e48d033d723d953a0a977bea14c94466e80c93d2e"
Oct 30 19:42:53.761793 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.761773    5820 scope.go:122] "RemoveContainer" containerID="12f9d851304b567522173c6e48d033d723d953a0a977bea14c94466e80c93d2e"
Oct 30 19:42:53.762111 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:42:53.762089    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12f9d851304b567522173c6e48d033d723d953a0a977bea14c94466e80c93d2e\": container with ID starting with 12f9d851304b567522173c6e48d033d723d953a0a977bea14c94466e80c93d2e not found: ID does not exist" containerID="12f9d851304b567522173c6e48d033d723d953a0a977bea14c94466e80c93d2e"
Oct 30 19:42:53.762169 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.762120    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12f9d851304b567522173c6e48d033d723d953a0a977bea14c94466e80c93d2e"} err="failed to get container status \"12f9d851304b567522173c6e48d033d723d953a0a977bea14c94466e80c93d2e\": rpc error: code = NotFound desc = could not find container \"12f9d851304b567522173c6e48d033d723d953a0a977bea14c94466e80c93d2e\": container with ID starting with 12f9d851304b567522173c6e48d033d723d953a0a977bea14c94466e80c93d2e not found: ID does not exist"
Oct 30 19:42:53.917149 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.917115    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"]
Oct 30 19:42:53.966362 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:53.966326    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-fd61e3/distinct-2-w1-fd61e3-kserve-769f676c64-rqt8n"]
Oct 30 19:42:55.436428 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:55.436397    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="80773b82-666e-428b-aeec-361a6c9cd6ac" path="/var/lib/kubelet/pods/80773b82-666e-428b-aeec-361a6c9cd6ac/volumes"
Oct 30 19:42:56.201872 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:56.201833    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw" podUID="c01e0998-f450-41a4-be3f-c9a94ea6ff59" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:42:56.202118 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:56.202091    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx" podUID="b576ab89-f39f-49d9-964b-333d324a9c3d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:42:56.865818 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:56.865775    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:57.452375 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:57.452330    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:57.633646 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:42:57.633611    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:01.270194 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.270162    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"]
Oct 30 19:43:01.270674 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.270356    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr" podUID="d10b9529-eef8-4269-b328-8475335d7ddb" containerName="maas-api" containerID="cri-o://1a7ba6841253fac6770da89edfdb87bd53861d054bef3adfe13cd0913fd73dfc" gracePeriod=30
Oct 30 19:43:01.418432 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.418396    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:01.521680 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.521624    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"
Oct 30 19:43:01.596517 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.596484    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d10b9529-eef8-4269-b328-8475335d7ddb-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d10b9529-eef8-4269-b328-8475335d7ddb-openshift-service-ca\") pod \"d10b9529-eef8-4269-b328-8475335d7ddb\" (UID: \"d10b9529-eef8-4269-b328-8475335d7ddb\") "
Oct 30 19:43:01.596714 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.596541    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d10b9529-eef8-4269-b328-8475335d7ddb-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d10b9529-eef8-4269-b328-8475335d7ddb-maas-api-tls\") pod \"d10b9529-eef8-4269-b328-8475335d7ddb\" (UID: \"d10b9529-eef8-4269-b328-8475335d7ddb\") "
Oct 30 19:43:01.596714 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.596612    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d10b9529-eef8-4269-b328-8475335d7ddb-kube-api-access-n2454\" (UniqueName: \"kubernetes.io/projected/d10b9529-eef8-4269-b328-8475335d7ddb-kube-api-access-n2454\") pod \"d10b9529-eef8-4269-b328-8475335d7ddb\" (UID: \"d10b9529-eef8-4269-b328-8475335d7ddb\") "
Oct 30 19:43:01.596714 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.596640    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d10b9529-eef8-4269-b328-8475335d7ddb-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d10b9529-eef8-4269-b328-8475335d7ddb-metrics-certs\") pod \"d10b9529-eef8-4269-b328-8475335d7ddb\" (UID: \"d10b9529-eef8-4269-b328-8475335d7ddb\") "
Oct 30 19:43:01.596924 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.596905    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d10b9529-eef8-4269-b328-8475335d7ddb-openshift-service-ca" pod "d10b9529-eef8-4269-b328-8475335d7ddb" (UID: "d10b9529-eef8-4269-b328-8475335d7ddb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:01.597062 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.597046    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d10b9529-eef8-4269-b328-8475335d7ddb-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:01.598893 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.598863    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d10b9529-eef8-4269-b328-8475335d7ddb-maas-api-tls" pod "d10b9529-eef8-4269-b328-8475335d7ddb" (UID: "d10b9529-eef8-4269-b328-8475335d7ddb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:43:01.598893 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.598871    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d10b9529-eef8-4269-b328-8475335d7ddb-kube-api-access-n2454" pod "d10b9529-eef8-4269-b328-8475335d7ddb" (UID: "d10b9529-eef8-4269-b328-8475335d7ddb"). InnerVolumeSpecName "kube-api-access-n2454". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:01.599077 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.598970    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d10b9529-eef8-4269-b328-8475335d7ddb-metrics-certs" pod "d10b9529-eef8-4269-b328-8475335d7ddb" (UID: "d10b9529-eef8-4269-b328-8475335d7ddb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:43:01.698249 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.698216    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d10b9529-eef8-4269-b328-8475335d7ddb-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:01.698249 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.698243    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2454\" (UniqueName: \"kubernetes.io/projected/d10b9529-eef8-4269-b328-8475335d7ddb-kube-api-access-n2454\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:01.698249 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.698254    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d10b9529-eef8-4269-b328-8475335d7ddb-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:01.799479 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.799449    5820 generic.go:356] "Generic (PLEG): container finished" podID="d10b9529-eef8-4269-b328-8475335d7ddb" containerID="1a7ba6841253fac6770da89edfdb87bd53861d054bef3adfe13cd0913fd73dfc" exitCode=0
Oct 30 19:43:01.799664 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.799498    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"
Oct 30 19:43:01.799664 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.799535    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr" event={"ID":"d10b9529-eef8-4269-b328-8475335d7ddb","Type":"ContainerDied","Data":"1a7ba6841253fac6770da89edfdb87bd53861d054bef3adfe13cd0913fd73dfc"}
Oct 30 19:43:01.799664 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.799593    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr" event={"ID":"d10b9529-eef8-4269-b328-8475335d7ddb","Type":"ContainerDied","Data":"071e57b2d954362e6ea4137fa61e655cfe2a3e0b6cc82bb29378bc7b92ce4f1a"}
Oct 30 19:43:01.799664 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.799619    5820 scope.go:122] "RemoveContainer" containerID="1a7ba6841253fac6770da89edfdb87bd53861d054bef3adfe13cd0913fd73dfc"
Oct 30 19:43:01.809544 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.809527    5820 scope.go:122] "RemoveContainer" containerID="1a7ba6841253fac6770da89edfdb87bd53861d054bef3adfe13cd0913fd73dfc"
Oct 30 19:43:01.809816 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:43:01.809791    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a7ba6841253fac6770da89edfdb87bd53861d054bef3adfe13cd0913fd73dfc\": container with ID starting with 1a7ba6841253fac6770da89edfdb87bd53861d054bef3adfe13cd0913fd73dfc not found: ID does not exist" containerID="1a7ba6841253fac6770da89edfdb87bd53861d054bef3adfe13cd0913fd73dfc"
Oct 30 19:43:01.809863 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.809819    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a7ba6841253fac6770da89edfdb87bd53861d054bef3adfe13cd0913fd73dfc"} err="failed to get container status \"1a7ba6841253fac6770da89edfdb87bd53861d054bef3adfe13cd0913fd73dfc\": rpc error: code = NotFound desc = could not find container \"1a7ba6841253fac6770da89edfdb87bd53861d054bef3adfe13cd0913fd73dfc\": container with ID starting with 1a7ba6841253fac6770da89edfdb87bd53861d054bef3adfe13cd0913fd73dfc not found: ID does not exist"
Oct 30 19:43:01.967827 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.967792    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"]
Oct 30 19:43:01.969885 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:01.969864    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-4a7a52-dc8676775-z4rcr"]
Oct 30 19:43:02.732126 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.732083    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-792a70a1-l4ncr"]
Oct 30 19:43:02.732713 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.732693    5820 cpu_manager.go:414] "containerMap: removing container" podUID="c1720171-99ee-4be0-8bbe-f8fdfed25319" containerName="istio-proxy"
Oct 30 19:43:02.732792 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.732714    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c1720171-99ee-4be0-8bbe-f8fdfed25319" containerName="istio-proxy"
Oct 30 19:43:02.732792 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.732733    5820 cpu_manager.go:414] "containerMap: removing container" podUID="f4415a3e-5238-48e7-a6b2-e97c1904381f" containerName="istio-proxy"
Oct 30 19:43:02.732792 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.732738    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4415a3e-5238-48e7-a6b2-e97c1904381f" containerName="istio-proxy"
Oct 30 19:43:02.732792 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.732761    5820 cpu_manager.go:414] "containerMap: removing container" podUID="d10b9529-eef8-4269-b328-8475335d7ddb" containerName="maas-api"
Oct 30 19:43:02.732792 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.732765    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d10b9529-eef8-4269-b328-8475335d7ddb" containerName="maas-api"
Oct 30 19:43:02.732792 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.732781    5820 cpu_manager.go:414] "containerMap: removing container" podUID="80773b82-666e-428b-aeec-361a6c9cd6ac" containerName="main"
Oct 30 19:43:02.732792 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.732785    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="80773b82-666e-428b-aeec-361a6c9cd6ac" containerName="main"
Oct 30 19:43:02.733046 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.732848    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="d10b9529-eef8-4269-b328-8475335d7ddb" containerName="maas-api"
Oct 30 19:43:02.733046 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.732857    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="80773b82-666e-428b-aeec-361a6c9cd6ac" containerName="main"
Oct 30 19:43:02.737416 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.737395    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-792a70a1-l4ncr"
Oct 30 19:43:02.794733 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.794703    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:43:02.794733 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.794730    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qg7n\""
Oct 30 19:43:02.797513 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.797490    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-792a70a1-l4ncr"]
Oct 30 19:43:02.809702 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.809671    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/52e18e09-877a-4bd7-971d-90d489e5102a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-792a70a1-l4ncr\" (UID: \"52e18e09-877a-4bd7-971d-90d489e5102a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-792a70a1-l4ncr"
Oct 30 19:43:02.809867 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.809783    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/52e18e09-877a-4bd7-971d-90d489e5102a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-792a70a1-l4ncr\" (UID: \"52e18e09-877a-4bd7-971d-90d489e5102a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-792a70a1-l4ncr"
Oct 30 19:43:02.809867 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.809847    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnt2v\" (UniqueName: \"kubernetes.io/projected/52e18e09-877a-4bd7-971d-90d489e5102a-kube-api-access-hnt2v\") pod \"maas-api-revoke-sub-792a70a1-l4ncr\" (UID: \"52e18e09-877a-4bd7-971d-90d489e5102a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-792a70a1-l4ncr"
Oct 30 19:43:02.911195 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.911159    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/52e18e09-877a-4bd7-971d-90d489e5102a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-792a70a1-l4ncr\" (UID: \"52e18e09-877a-4bd7-971d-90d489e5102a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-792a70a1-l4ncr"
Oct 30 19:43:02.911398 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.911229    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/52e18e09-877a-4bd7-971d-90d489e5102a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-792a70a1-l4ncr\" (UID: \"52e18e09-877a-4bd7-971d-90d489e5102a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-792a70a1-l4ncr"
Oct 30 19:43:02.911398 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.911286    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnt2v\" (UniqueName: \"kubernetes.io/projected/52e18e09-877a-4bd7-971d-90d489e5102a-kube-api-access-hnt2v\") pod \"maas-api-revoke-sub-792a70a1-l4ncr\" (UID: \"52e18e09-877a-4bd7-971d-90d489e5102a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-792a70a1-l4ncr"
Oct 30 19:43:02.911842 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.911819    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/52e18e09-877a-4bd7-971d-90d489e5102a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-792a70a1-l4ncr\" (UID: \"52e18e09-877a-4bd7-971d-90d489e5102a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-792a70a1-l4ncr"
Oct 30 19:43:02.913676 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.913656    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/52e18e09-877a-4bd7-971d-90d489e5102a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-792a70a1-l4ncr\" (UID: \"52e18e09-877a-4bd7-971d-90d489e5102a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-792a70a1-l4ncr"
Oct 30 19:43:02.972768 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:02.972727    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnt2v\" (UniqueName: \"kubernetes.io/projected/52e18e09-877a-4bd7-971d-90d489e5102a-kube-api-access-hnt2v\") pod \"maas-api-revoke-sub-792a70a1-l4ncr\" (UID: \"52e18e09-877a-4bd7-971d-90d489e5102a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-792a70a1-l4ncr"
Oct 30 19:43:03.054103 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:03.054075    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-792a70a1-l4ncr"
Oct 30 19:43:03.182535 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:03.182500    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:03.428033 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:03.428005    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-792a70a1-l4ncr"]
Oct 30 19:43:03.431082 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:43:03.431056    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52e18e09_877a_4bd7_971d_90d489e5102a.slice/crio-3b1258f48a977edb73213c33aa0db3018b9d237d735f388bd28ec5e38a65ef4c WatchSource:0}: Error finding container 3b1258f48a977edb73213c33aa0db3018b9d237d735f388bd28ec5e38a65ef4c: Status 404 returned error can't find the container with id 3b1258f48a977edb73213c33aa0db3018b9d237d735f388bd28ec5e38a65ef4c
Oct 30 19:43:03.438345 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:03.438313    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d10b9529-eef8-4269-b328-8475335d7ddb" path="/var/lib/kubelet/pods/d10b9529-eef8-4269-b328-8475335d7ddb/volumes"
Oct 30 19:43:03.483874 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:03.483838    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:03.688394 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:03.688318    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:03.814846 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:03.814811    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-792a70a1-l4ncr" event={"ID":"52e18e09-877a-4bd7-971d-90d489e5102a","Type":"ContainerStarted","Data":"2f2d554f19302f10d1dac79fe1f83c15cda3ce8fe3957bfcfc87ca6e2329befa"}
Oct 30 19:43:03.815303 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:03.814866    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-792a70a1-l4ncr" event={"ID":"52e18e09-877a-4bd7-971d-90d489e5102a","Type":"ContainerStarted","Data":"3b1258f48a977edb73213c33aa0db3018b9d237d735f388bd28ec5e38a65ef4c"}
Oct 30 19:43:03.931669 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:03.931613    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-792a70a1-l4ncr" podStartSLOduration=1.931595011 podStartE2EDuration="1.931595011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:03.930363567 +0000 UTC m=+4104.642112265" watchObservedRunningTime="2026-10-30 19:43:03.931595011 +0000 UTC m=+4104.643343708"
Oct 30 19:43:04.152774 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:04.152731    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:04.822519 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:04.822481    5820 generic.go:356] "Generic (PLEG): container finished" podID="52e18e09-877a-4bd7-971d-90d489e5102a" containerID="2f2d554f19302f10d1dac79fe1f83c15cda3ce8fe3957bfcfc87ca6e2329befa" exitCode=0
Oct 30 19:43:04.822924 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:04.822561    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-792a70a1-l4ncr" event={"ID":"52e18e09-877a-4bd7-971d-90d489e5102a","Type":"ContainerDied","Data":"2f2d554f19302f10d1dac79fe1f83c15cda3ce8fe3957bfcfc87ca6e2329befa"}
Oct 30 19:43:05.953266 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:05.953237    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-792a70a1-l4ncr"
Oct 30 19:43:06.044871 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:06.044791    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:06.048476 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:06.048445    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/52e18e09-877a-4bd7-971d-90d489e5102a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/52e18e09-877a-4bd7-971d-90d489e5102a-maas-api-openshift-ca\") pod \"52e18e09-877a-4bd7-971d-90d489e5102a\" (UID: \"52e18e09-877a-4bd7-971d-90d489e5102a\") "
Oct 30 19:43:06.048623 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:06.048565    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/52e18e09-877a-4bd7-971d-90d489e5102a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/52e18e09-877a-4bd7-971d-90d489e5102a-maas-api-xks-ca\") pod \"52e18e09-877a-4bd7-971d-90d489e5102a\" (UID: \"52e18e09-877a-4bd7-971d-90d489e5102a\") "
Oct 30 19:43:06.048703 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:06.048658    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52e18e09-877a-4bd7-971d-90d489e5102a-kube-api-access-hnt2v\" (UniqueName: \"kubernetes.io/projected/52e18e09-877a-4bd7-971d-90d489e5102a-kube-api-access-hnt2v\") pod \"52e18e09-877a-4bd7-971d-90d489e5102a\" (UID: \"52e18e09-877a-4bd7-971d-90d489e5102a\") "
Oct 30 19:43:06.048810 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:06.048787    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52e18e09-877a-4bd7-971d-90d489e5102a-maas-api-openshift-ca" pod "52e18e09-877a-4bd7-971d-90d489e5102a" (UID: "52e18e09-877a-4bd7-971d-90d489e5102a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:06.049035 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:06.049009    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/52e18e09-877a-4bd7-971d-90d489e5102a-maas-api-openshift-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:06.050907 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:06.050880    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52e18e09-877a-4bd7-971d-90d489e5102a-maas-api-xks-ca" pod "52e18e09-877a-4bd7-971d-90d489e5102a" (UID: "52e18e09-877a-4bd7-971d-90d489e5102a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:43:06.051028 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:06.050904    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52e18e09-877a-4bd7-971d-90d489e5102a-kube-api-access-hnt2v" pod "52e18e09-877a-4bd7-971d-90d489e5102a" (UID: "52e18e09-877a-4bd7-971d-90d489e5102a"). InnerVolumeSpecName "kube-api-access-hnt2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:06.150493 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:06.150417    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/52e18e09-877a-4bd7-971d-90d489e5102a-maas-api-xks-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:06.150493 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:06.150443    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnt2v\" (UniqueName: \"kubernetes.io/projected/52e18e09-877a-4bd7-971d-90d489e5102a-kube-api-access-hnt2v\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:06.200607 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:06.200572    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw" podUID="c01e0998-f450-41a4-be3f-c9a94ea6ff59" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:43:06.201892 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:06.201862    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx" podUID="b576ab89-f39f-49d9-964b-333d324a9c3d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:43:06.833612 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:06.833574    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-792a70a1-l4ncr" event={"ID":"52e18e09-877a-4bd7-971d-90d489e5102a","Type":"ContainerDied","Data":"3b1258f48a977edb73213c33aa0db3018b9d237d735f388bd28ec5e38a65ef4c"}
Oct 30 19:43:06.833799 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:06.833624    5820 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b1258f48a977edb73213c33aa0db3018b9d237d735f388bd28ec5e38a65ef4c"
Oct 30 19:43:06.833799 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:06.833583    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-792a70a1-l4ncr"
Oct 30 19:43:07.327470 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:07.327441    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:07.532914 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:07.532881    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:07.723541 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:07.723449    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:07.956267 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:07.956231    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:08.133831 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:08.133800    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:08.203319 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:43:08.203283    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:43:08.641923 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:08.641882    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:08.993273 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:08.993183    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:09.194737 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:09.194676    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:09.944147 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:09.944113    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:10.256388 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.256364    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx"
Oct 30 19:43:10.371294 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.371270    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw"
Oct 30 19:43:10.392838 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.392809    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b576ab89-f39f-49d9-964b-333d324a9c3d-kube-api-access-vwv6h\" (UniqueName: \"kubernetes.io/projected/b576ab89-f39f-49d9-964b-333d324a9c3d-kube-api-access-vwv6h\") pod \"b576ab89-f39f-49d9-964b-333d324a9c3d\" (UID: \"b576ab89-f39f-49d9-964b-333d324a9c3d\") "
Oct 30 19:43:10.392977 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.392875    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b576ab89-f39f-49d9-964b-333d324a9c3d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b576ab89-f39f-49d9-964b-333d324a9c3d-plugins-config-volume\") pod \"b576ab89-f39f-49d9-964b-333d324a9c3d\" (UID: \"b576ab89-f39f-49d9-964b-333d324a9c3d\") "
Oct 30 19:43:10.393285 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.393263    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b576ab89-f39f-49d9-964b-333d324a9c3d-plugins-config-volume" pod "b576ab89-f39f-49d9-964b-333d324a9c3d" (UID: "b576ab89-f39f-49d9-964b-333d324a9c3d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:10.395017 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.394995    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b576ab89-f39f-49d9-964b-333d324a9c3d-kube-api-access-vwv6h" pod "b576ab89-f39f-49d9-964b-333d324a9c3d" (UID: "b576ab89-f39f-49d9-964b-333d324a9c3d"). InnerVolumeSpecName "kube-api-access-vwv6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:10.494103 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.494026    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c01e0998-f450-41a4-be3f-c9a94ea6ff59-kube-api-access-wnwtv\" (UniqueName: \"kubernetes.io/projected/c01e0998-f450-41a4-be3f-c9a94ea6ff59-kube-api-access-wnwtv\") pod \"c01e0998-f450-41a4-be3f-c9a94ea6ff59\" (UID: \"c01e0998-f450-41a4-be3f-c9a94ea6ff59\") "
Oct 30 19:43:10.494265 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.494173    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c01e0998-f450-41a4-be3f-c9a94ea6ff59-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c01e0998-f450-41a4-be3f-c9a94ea6ff59-plugins-config-volume\") pod \"c01e0998-f450-41a4-be3f-c9a94ea6ff59\" (UID: \"c01e0998-f450-41a4-be3f-c9a94ea6ff59\") "
Oct 30 19:43:10.494494 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.494476    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b576ab89-f39f-49d9-964b-333d324a9c3d-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:10.494561 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.494493    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c01e0998-f450-41a4-be3f-c9a94ea6ff59-plugins-config-volume" pod "c01e0998-f450-41a4-be3f-c9a94ea6ff59" (UID: "c01e0998-f450-41a4-be3f-c9a94ea6ff59"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:10.494561 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.494499    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwv6h\" (UniqueName: \"kubernetes.io/projected/b576ab89-f39f-49d9-964b-333d324a9c3d-kube-api-access-vwv6h\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:10.496299 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.496275    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c01e0998-f450-41a4-be3f-c9a94ea6ff59-kube-api-access-wnwtv" pod "c01e0998-f450-41a4-be3f-c9a94ea6ff59" (UID: "c01e0998-f450-41a4-be3f-c9a94ea6ff59"). InnerVolumeSpecName "kube-api-access-wnwtv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:10.596000 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.595945    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnwtv\" (UniqueName: \"kubernetes.io/projected/c01e0998-f450-41a4-be3f-c9a94ea6ff59-kube-api-access-wnwtv\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:10.596000 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.595997    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c01e0998-f450-41a4-be3f-c9a94ea6ff59-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:10.855926 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.855886    5820 generic.go:356] "Generic (PLEG): container finished" podID="b576ab89-f39f-49d9-964b-333d324a9c3d" containerID="7f959ec52dc1a86d1c9688db7f0b34c5af5015db283923081a456259c3445368" exitCode=0
Oct 30 19:43:10.856147 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.855938    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx"
Oct 30 19:43:10.856147 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.855974    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx" event={"ID":"b576ab89-f39f-49d9-964b-333d324a9c3d","Type":"ContainerDied","Data":"7f959ec52dc1a86d1c9688db7f0b34c5af5015db283923081a456259c3445368"}
Oct 30 19:43:10.856147 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.856027    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx" event={"ID":"b576ab89-f39f-49d9-964b-333d324a9c3d","Type":"ContainerDied","Data":"a7255ddf87efc06dafb6eb57c53814752d98bed66a29a4073862b6a7375dccfa"}
Oct 30 19:43:10.856147 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.856049    5820 scope.go:122] "RemoveContainer" containerID="7f959ec52dc1a86d1c9688db7f0b34c5af5015db283923081a456259c3445368"
Oct 30 19:43:10.857847 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.857659    5820 generic.go:356] "Generic (PLEG): container finished" podID="c01e0998-f450-41a4-be3f-c9a94ea6ff59" containerID="9f1993aab2fff61086fab35dd9129d09178456f55764b94d5753e692642bd072" exitCode=0
Oct 30 19:43:10.857847 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.857749    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw"
Oct 30 19:43:10.857847 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.857769    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw" event={"ID":"c01e0998-f450-41a4-be3f-c9a94ea6ff59","Type":"ContainerDied","Data":"9f1993aab2fff61086fab35dd9129d09178456f55764b94d5753e692642bd072"}
Oct 30 19:43:10.857847 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.857812    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw" event={"ID":"c01e0998-f450-41a4-be3f-c9a94ea6ff59","Type":"ContainerDied","Data":"845396b929b964c935a2b52f8f7b8e8bd2eca0b5a6a828e03f77ccfb9e421ee0"}
Oct 30 19:43:10.865695 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.865680    5820 scope.go:122] "RemoveContainer" containerID="7f959ec52dc1a86d1c9688db7f0b34c5af5015db283923081a456259c3445368"
Oct 30 19:43:10.866023 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:43:10.865998    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f959ec52dc1a86d1c9688db7f0b34c5af5015db283923081a456259c3445368\": container with ID starting with 7f959ec52dc1a86d1c9688db7f0b34c5af5015db283923081a456259c3445368 not found: ID does not exist" containerID="7f959ec52dc1a86d1c9688db7f0b34c5af5015db283923081a456259c3445368"
Oct 30 19:43:10.866103 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.866036    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f959ec52dc1a86d1c9688db7f0b34c5af5015db283923081a456259c3445368"} err="failed to get container status \"7f959ec52dc1a86d1c9688db7f0b34c5af5015db283923081a456259c3445368\": rpc error: code = NotFound desc = could not find container \"7f959ec52dc1a86d1c9688db7f0b34c5af5015db283923081a456259c3445368\": container with ID starting with 7f959ec52dc1a86d1c9688db7f0b34c5af5015db283923081a456259c3445368 not found: ID does not exist"
Oct 30 19:43:10.866103 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.866060    5820 scope.go:122] "RemoveContainer" containerID="9f1993aab2fff61086fab35dd9129d09178456f55764b94d5753e692642bd072"
Oct 30 19:43:10.868720 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.868695    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:10.874697 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.874679    5820 scope.go:122] "RemoveContainer" containerID="9f1993aab2fff61086fab35dd9129d09178456f55764b94d5753e692642bd072"
Oct 30 19:43:10.874964 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:43:10.874931    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f1993aab2fff61086fab35dd9129d09178456f55764b94d5753e692642bd072\": container with ID starting with 9f1993aab2fff61086fab35dd9129d09178456f55764b94d5753e692642bd072 not found: ID does not exist" containerID="9f1993aab2fff61086fab35dd9129d09178456f55764b94d5753e692642bd072"
Oct 30 19:43:10.875011 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:10.874968    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f1993aab2fff61086fab35dd9129d09178456f55764b94d5753e692642bd072"} err="failed to get container status \"9f1993aab2fff61086fab35dd9129d09178456f55764b94d5753e692642bd072\": rpc error: code = NotFound desc = could not find container \"9f1993aab2fff61086fab35dd9129d09178456f55764b94d5753e692642bd072\": container with ID starting with 9f1993aab2fff61086fab35dd9129d09178456f55764b94d5753e692642bd072 not found: ID does not exist"
Oct 30 19:43:11.064543 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:11.064509    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw"]
Oct 30 19:43:11.096695 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:11.096663    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:11.113608 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:11.113549    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-fd61e3-7d8c9cbfb7-qqwsw"]
Oct 30 19:43:11.189413 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:11.189377    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx"]
Oct 30 19:43:11.194227 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:11.194201    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-fd61e3-65dc675c58-ggdlx"]
Oct 30 19:43:11.277392 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:11.277361    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:11.436488 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:11.436401    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b576ab89-f39f-49d9-964b-333d324a9c3d" path="/var/lib/kubelet/pods/b576ab89-f39f-49d9-964b-333d324a9c3d/volumes"
Oct 30 19:43:11.436884 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:11.436862    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c01e0998-f450-41a4-be3f-c9a94ea6ff59" path="/var/lib/kubelet/pods/c01e0998-f450-41a4-be3f-c9a94ea6ff59/volumes"
Oct 30 19:43:12.765992 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:12.765932    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:12.954914 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:12.954879    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:13.244837 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:13.244808    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:13.462338 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:13.462302    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:14.343943 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.343908    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"]
Oct 30 19:43:14.344412 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.344398    5820 cpu_manager.go:414] "containerMap: removing container" podUID="c01e0998-f450-41a4-be3f-c9a94ea6ff59" containerName="payload-pre-processing"
Oct 30 19:43:14.344470 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.344414    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c01e0998-f450-41a4-be3f-c9a94ea6ff59" containerName="payload-pre-processing"
Oct 30 19:43:14.344470 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.344425    5820 cpu_manager.go:414] "containerMap: removing container" podUID="52e18e09-877a-4bd7-971d-90d489e5102a" containerName="revoke-keys"
Oct 30 19:43:14.344470 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.344431    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="52e18e09-877a-4bd7-971d-90d489e5102a" containerName="revoke-keys"
Oct 30 19:43:14.344470 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.344441    5820 cpu_manager.go:414] "containerMap: removing container" podUID="b576ab89-f39f-49d9-964b-333d324a9c3d" containerName="payload-processing"
Oct 30 19:43:14.344470 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.344446    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b576ab89-f39f-49d9-964b-333d324a9c3d" containerName="payload-processing"
Oct 30 19:43:14.344630 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.344506    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="c01e0998-f450-41a4-be3f-c9a94ea6ff59" containerName="payload-pre-processing"
Oct 30 19:43:14.344630 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.344515    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="52e18e09-877a-4bd7-971d-90d489e5102a" containerName="revoke-keys"
Oct 30 19:43:14.344630 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.344523    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="b576ab89-f39f-49d9-964b-333d324a9c3d" containerName="payload-processing"
Oct 30 19:43:14.347516 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.347499    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"
Oct 30 19:43:14.399817 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.399785    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-5da1ae\"/\"payload-processing-external-model-plugins-e2e-extmod-b-5da1ae\""
Oct 30 19:43:14.449110 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.449075    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-5da1ae\"/\"openshift-service-ca.crt\""
Oct 30 19:43:14.449110 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.449075    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-5da1ae\"/\"routing-overlay\""
Oct 30 19:43:14.449365 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.449112    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-5da1ae\"/\"kube-root-ca.crt\""
Oct 30 19:43:14.449365 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.449126    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-5da1ae\"/\"agc-cred-3fe404dba993\""
Oct 30 19:43:14.449365 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.449190    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-5da1ae\"/\"payload-processing-external-model-e2e--cd7b9eef-dockercfg-9qd4n\""
Oct 30 19:43:14.451208 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.451190    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"]
Oct 30 19:43:14.535454 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.535412    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5771fcc-1572-4603-afc0-90fd231b731f-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926\" (UID: \"e5771fcc-1572-4603-afc0-90fd231b731f\") " pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"
Oct 30 19:43:14.535651 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.535505    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e5771fcc-1572-4603-afc0-90fd231b731f-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926\" (UID: \"e5771fcc-1572-4603-afc0-90fd231b731f\") " pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"
Oct 30 19:43:14.535651 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.535535    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e5771fcc-1572-4603-afc0-90fd231b731f-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926\" (UID: \"e5771fcc-1572-4603-afc0-90fd231b731f\") " pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"
Oct 30 19:43:14.535651 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.535617    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgj4n\" (UniqueName: \"kubernetes.io/projected/e5771fcc-1572-4603-afc0-90fd231b731f-kube-api-access-xgj4n\") pod \"payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926\" (UID: \"e5771fcc-1572-4603-afc0-90fd231b731f\") " pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"
Oct 30 19:43:14.636785 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.636700    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5771fcc-1572-4603-afc0-90fd231b731f-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926\" (UID: \"e5771fcc-1572-4603-afc0-90fd231b731f\") " pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"
Oct 30 19:43:14.636785 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.636743    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e5771fcc-1572-4603-afc0-90fd231b731f-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926\" (UID: \"e5771fcc-1572-4603-afc0-90fd231b731f\") " pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"
Oct 30 19:43:14.636785 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.636764    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e5771fcc-1572-4603-afc0-90fd231b731f-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926\" (UID: \"e5771fcc-1572-4603-afc0-90fd231b731f\") " pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"
Oct 30 19:43:14.636785 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.636782    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgj4n\" (UniqueName: \"kubernetes.io/projected/e5771fcc-1572-4603-afc0-90fd231b731f-kube-api-access-xgj4n\") pod \"payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926\" (UID: \"e5771fcc-1572-4603-afc0-90fd231b731f\") " pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"
Oct 30 19:43:14.637400 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.637370    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e5771fcc-1572-4603-afc0-90fd231b731f-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926\" (UID: \"e5771fcc-1572-4603-afc0-90fd231b731f\") " pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"
Oct 30 19:43:14.637605 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.637584    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5771fcc-1572-4603-afc0-90fd231b731f-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926\" (UID: \"e5771fcc-1572-4603-afc0-90fd231b731f\") " pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"
Oct 30 19:43:14.639499 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.639476    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e5771fcc-1572-4603-afc0-90fd231b731f-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926\" (UID: \"e5771fcc-1572-4603-afc0-90fd231b731f\") " pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"
Oct 30 19:43:14.697630 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.697600    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgj4n\" (UniqueName: \"kubernetes.io/projected/e5771fcc-1572-4603-afc0-90fd231b731f-kube-api-access-xgj4n\") pod \"payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926\" (UID: \"e5771fcc-1572-4603-afc0-90fd231b731f\") " pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"
Oct 30 19:43:14.960443 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:14.960356    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"
Oct 30 19:43:15.333370 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:15.333292    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"]
Oct 30 19:43:15.336197 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:43:15.336158    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5771fcc_1572_4603_afc0_90fd231b731f.slice/crio-e63069d2d48c469f3b55ee7c2ee7373ee1267c15a3a52195306e98aa462ef008 WatchSource:0}: Error finding container e63069d2d48c469f3b55ee7c2ee7373ee1267c15a3a52195306e98aa462ef008: Status 404 returned error can't find the container with id e63069d2d48c469f3b55ee7c2ee7373ee1267c15a3a52195306e98aa462ef008
Oct 30 19:43:15.570651 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:15.570617    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:15.774647 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:15.774563    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:15.889039 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:15.888997    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926" event={"ID":"e5771fcc-1572-4603-afc0-90fd231b731f","Type":"ContainerStarted","Data":"9f4d89bac20a4a481094bfc57a0927f77ae8c4c838348a8fd57c369017eabbb1"}
Oct 30 19:43:15.889226 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:15.889062    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926" event={"ID":"e5771fcc-1572-4603-afc0-90fd231b731f","Type":"ContainerStarted","Data":"e63069d2d48c469f3b55ee7c2ee7373ee1267c15a3a52195306e98aa462ef008"}
Oct 30 19:43:15.889226 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:15.889182    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"
Oct 30 19:43:16.007604 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:16.007546    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926" podStartSLOduration=2.007531783 podStartE2EDuration="2.007531783s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:16.007138925 +0000 UTC m=+4116.718887623" watchObservedRunningTime="2026-10-30 19:43:16.007531783 +0000 UTC m=+4116.719280481"
Oct 30 19:43:16.328490 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:16.328453    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:18.364170 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:18.364132    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:22.596477 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:22.596425    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:22.956534 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:22.956432    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:23.184282 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:23.184240    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:23.761091 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:23.761047    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:24.180344 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:24.180297    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:24.433459 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:24.433383    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:25.068436 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:25.068393    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:25.525140 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:25.525060    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:25.714860 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:25.714815    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:26.387551 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:26.387512    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:26.896850 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:26.896824    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"
Oct 30 19:43:27.494020 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:27.493981    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:27.676066 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:27.676015    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:28.305136 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:28.305083    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:30.738177 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:30.738143    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:32.057014 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:32.056975    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:32.236833 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:32.236801    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:32.428272 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:32.428193    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:33.545109 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:33.545075    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:33.745062 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:33.745032    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:33.931288 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:33.929770    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:34.427624 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:34.427582    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:35.666174 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:35.666121    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:38.202161 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:43:38.202127    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:43:40.658178 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:40.658136    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:45.064783 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:45.064745    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:45.641294 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:45.641256    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:50.474195 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:50.474105    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:52.477683 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:52.477648    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:55.343846 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:43:55.343799    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:00.277280 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:00.277239    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:05.237486 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:05.237443    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:08.201971 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:44:08.201913    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:44:10.069396 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:10.069346    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:10.311712 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:10.311654    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:15.040729 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:15.040691    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:19.975279 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:19.975241    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:22.504671 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:22.504636    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"]
Oct 30 19:44:22.550586 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:22.550551    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:22.761651 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:22.761569    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:23.437676 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:23.437639    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:24.961924 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:24.961883    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:27.505376 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:27.505325    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926" podUID="e5771fcc-1572-4603-afc0-90fd231b731f" containerName="payload-processing" containerID="cri-o://9f4d89bac20a4a481094bfc57a0927f77ae8c4c838348a8fd57c369017eabbb1" gracePeriod=25
Oct 30 19:44:27.765438 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:27.765377    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"
Oct 30 19:44:27.938144 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:27.938116    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e5771fcc-1572-4603-afc0-90fd231b731f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5771fcc-1572-4603-afc0-90fd231b731f-plugins-config-volume\") pod \"e5771fcc-1572-4603-afc0-90fd231b731f\" (UID: \"e5771fcc-1572-4603-afc0-90fd231b731f\") "
Oct 30 19:44:27.938319 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:27.938165    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5771fcc-1572-4603-afc0-90fd231b731f-kube-api-access-xgj4n\" (UniqueName: \"kubernetes.io/projected/e5771fcc-1572-4603-afc0-90fd231b731f-kube-api-access-xgj4n\") pod \"e5771fcc-1572-4603-afc0-90fd231b731f\" (UID: \"e5771fcc-1572-4603-afc0-90fd231b731f\") "
Oct 30 19:44:27.938319 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:27.938257    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5771fcc-1572-4603-afc0-90fd231b731f-provider-credentials\" (UniqueName: \"kubernetes.io/projected/e5771fcc-1572-4603-afc0-90fd231b731f-provider-credentials\") pod \"e5771fcc-1572-4603-afc0-90fd231b731f\" (UID: \"e5771fcc-1572-4603-afc0-90fd231b731f\") "
Oct 30 19:44:27.938319 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:27.938309    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e5771fcc-1572-4603-afc0-90fd231b731f-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e5771fcc-1572-4603-afc0-90fd231b731f-routing-overlay-volume\") pod \"e5771fcc-1572-4603-afc0-90fd231b731f\" (UID: \"e5771fcc-1572-4603-afc0-90fd231b731f\") "
Oct 30 19:44:27.938682 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:27.938555    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5771fcc-1572-4603-afc0-90fd231b731f-plugins-config-volume" pod "e5771fcc-1572-4603-afc0-90fd231b731f" (UID: "e5771fcc-1572-4603-afc0-90fd231b731f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:44:27.938784 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:27.938726    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5771fcc-1572-4603-afc0-90fd231b731f-routing-overlay-volume" pod "e5771fcc-1572-4603-afc0-90fd231b731f" (UID: "e5771fcc-1572-4603-afc0-90fd231b731f"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:44:27.940587 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:27.940561    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5771fcc-1572-4603-afc0-90fd231b731f-provider-credentials" pod "e5771fcc-1572-4603-afc0-90fd231b731f" (UID: "e5771fcc-1572-4603-afc0-90fd231b731f"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:27.940681 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:27.940636    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5771fcc-1572-4603-afc0-90fd231b731f-kube-api-access-xgj4n" pod "e5771fcc-1572-4603-afc0-90fd231b731f" (UID: "e5771fcc-1572-4603-afc0-90fd231b731f"). InnerVolumeSpecName "kube-api-access-xgj4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:28.039338 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:28.039263    5820 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e5771fcc-1572-4603-afc0-90fd231b731f-routing-overlay-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:44:28.039338 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:28.039290    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5771fcc-1572-4603-afc0-90fd231b731f-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:44:28.039338 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:28.039302    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgj4n\" (UniqueName: \"kubernetes.io/projected/e5771fcc-1572-4603-afc0-90fd231b731f-kube-api-access-xgj4n\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:44:28.039338 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:28.039310    5820 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e5771fcc-1572-4603-afc0-90fd231b731f-provider-credentials\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:44:28.256465 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:28.256430    5820 generic.go:356] "Generic (PLEG): container finished" podID="e5771fcc-1572-4603-afc0-90fd231b731f" containerID="9f4d89bac20a4a481094bfc57a0927f77ae8c4c838348a8fd57c369017eabbb1" exitCode=0
Oct 30 19:44:28.256622 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:28.256464    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926" event={"ID":"e5771fcc-1572-4603-afc0-90fd231b731f","Type":"ContainerDied","Data":"9f4d89bac20a4a481094bfc57a0927f77ae8c4c838348a8fd57c369017eabbb1"}
Oct 30 19:44:28.256622 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:28.256491    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"
Oct 30 19:44:28.256622 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:28.256565    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926" event={"ID":"e5771fcc-1572-4603-afc0-90fd231b731f","Type":"ContainerDied","Data":"e63069d2d48c469f3b55ee7c2ee7373ee1267c15a3a52195306e98aa462ef008"}
Oct 30 19:44:28.256622 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:28.256597    5820 scope.go:122] "RemoveContainer" containerID="9f4d89bac20a4a481094bfc57a0927f77ae8c4c838348a8fd57c369017eabbb1"
Oct 30 19:44:28.266324 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:28.266305    5820 scope.go:122] "RemoveContainer" containerID="9f4d89bac20a4a481094bfc57a0927f77ae8c4c838348a8fd57c369017eabbb1"
Oct 30 19:44:28.266565 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:44:28.266547    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f4d89bac20a4a481094bfc57a0927f77ae8c4c838348a8fd57c369017eabbb1\": container with ID starting with 9f4d89bac20a4a481094bfc57a0927f77ae8c4c838348a8fd57c369017eabbb1 not found: ID does not exist" containerID="9f4d89bac20a4a481094bfc57a0927f77ae8c4c838348a8fd57c369017eabbb1"
Oct 30 19:44:28.266621 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:28.266573    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f4d89bac20a4a481094bfc57a0927f77ae8c4c838348a8fd57c369017eabbb1"} err="failed to get container status \"9f4d89bac20a4a481094bfc57a0927f77ae8c4c838348a8fd57c369017eabbb1\": rpc error: code = NotFound desc = could not find container \"9f4d89bac20a4a481094bfc57a0927f77ae8c4c838348a8fd57c369017eabbb1\": container with ID starting with 9f4d89bac20a4a481094bfc57a0927f77ae8c4c838348a8fd57c369017eabbb1 not found: ID does not exist"
Oct 30 19:44:28.422017 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:28.421989    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"]
Oct 30 19:44:28.471471 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:28.471445    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-5da1ae/payload-processing-external-model-e2e-extmod-b-5da1ae-57b9cf926"]
Oct 30 19:44:29.435444 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:29.435414    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e5771fcc-1572-4603-afc0-90fd231b731f" path="/var/lib/kubelet/pods/e5771fcc-1572-4603-afc0-90fd231b731f/volumes"
Oct 30 19:44:29.886964 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:29.886923    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:30.152237 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:30.152141    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:30.367526 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:30.367491    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:34.859971 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:34.859916    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:37.251790 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:37.251753    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:37.485561 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:37.485527    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:38.203261 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:44:38.203228    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:44:40.228891 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:40.228857    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:45.132381 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:45.132320    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:49.324715 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:49.324677    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:49.668351 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:49.668166    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:49.926028 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:49.925906    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:51.218913 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:51.218873    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:53.816420 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:53.816379    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:54.090440 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:54.090362    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:54.755298 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:54.755259    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:56.381938 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:44:56.381902    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:01.242059 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:01.242008    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:02.105582 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:02.105500    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:02.297827 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:02.297785    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:02.539985 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:02.539881    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:06.448706 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:06.448672    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:07.892056 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:07.892024    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:08.203912 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:45:08.203828    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:45:10.667029 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:10.666985    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:13.623074 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:13.623038    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:14.878849 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:14.878809    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw"]
Oct 30 19:45:15.003663 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:15.003624    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw"]
Oct 30 19:45:15.309597 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:15.309563    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:15.604327 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:15.604238    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:16.262115 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:16.262079    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:19.879653 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:19.879546    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw" podUID="420b7e12-f43e-45e0-be09-042d57313f5b" containerName="payload-processing" containerID="cri-o://526bdff887834236f7b34d96a7d1f48eb2feea0d4f73cd8615863fe75a894919" gracePeriod=25
Oct 30 19:45:20.004292 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:20.004248    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw" podUID="6211ab6d-429e-41e9-b0a0-3a1d76043165" containerName="payload-pre-processing" containerID="cri-o://c9b3f6b82afb57b0eb1faa281b1e25ce512b28a3a21dd09a7d320d3bdd339887" gracePeriod=25
Oct 30 19:45:20.361664 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:20.361625    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw" podUID="420b7e12-f43e-45e0-be09-042d57313f5b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:45:20.361851 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:20.361625    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw" podUID="6211ab6d-429e-41e9-b0a0-3a1d76043165" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:45:21.167055 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:21.167012    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:25.219043 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.218999    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"]
Oct 30 19:45:25.219529 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.219209    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck" podUID="3bb27649-8419-414f-a8f0-23a1838ea701" containerName="maas-api" containerID="cri-o://e276416a9b136f00bec4b107ecfb9ea9e2830da3c8267bf2b28e4404f488e5b8" gracePeriod=30
Oct 30 19:45:25.369651 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.369612    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:25.477705 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.477646    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"
Oct 30 19:45:25.530678 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.530643    5820 generic.go:356] "Generic (PLEG): container finished" podID="3bb27649-8419-414f-a8f0-23a1838ea701" containerID="e276416a9b136f00bec4b107ecfb9ea9e2830da3c8267bf2b28e4404f488e5b8" exitCode=0
Oct 30 19:45:25.530863 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.530688    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"
Oct 30 19:45:25.530863 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.530720    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck" event={"ID":"3bb27649-8419-414f-a8f0-23a1838ea701","Type":"ContainerDied","Data":"e276416a9b136f00bec4b107ecfb9ea9e2830da3c8267bf2b28e4404f488e5b8"}
Oct 30 19:45:25.530863 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.530771    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck" event={"ID":"3bb27649-8419-414f-a8f0-23a1838ea701","Type":"ContainerDied","Data":"6f9d039d4f36679241bef77f6056a403a0a173d3d0f7214fc98ddf0ced95492b"}
Oct 30 19:45:25.530863 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.530793    5820 scope.go:122] "RemoveContainer" containerID="e276416a9b136f00bec4b107ecfb9ea9e2830da3c8267bf2b28e4404f488e5b8"
Oct 30 19:45:25.539868 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.539840    5820 scope.go:122] "RemoveContainer" containerID="e276416a9b136f00bec4b107ecfb9ea9e2830da3c8267bf2b28e4404f488e5b8"
Oct 30 19:45:25.540177 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:45:25.540156    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e276416a9b136f00bec4b107ecfb9ea9e2830da3c8267bf2b28e4404f488e5b8\": container with ID starting with e276416a9b136f00bec4b107ecfb9ea9e2830da3c8267bf2b28e4404f488e5b8 not found: ID does not exist" containerID="e276416a9b136f00bec4b107ecfb9ea9e2830da3c8267bf2b28e4404f488e5b8"
Oct 30 19:45:25.540264 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.540184    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e276416a9b136f00bec4b107ecfb9ea9e2830da3c8267bf2b28e4404f488e5b8"} err="failed to get container status \"e276416a9b136f00bec4b107ecfb9ea9e2830da3c8267bf2b28e4404f488e5b8\": rpc error: code = NotFound desc = could not find container \"e276416a9b136f00bec4b107ecfb9ea9e2830da3c8267bf2b28e4404f488e5b8\": container with ID starting with e276416a9b136f00bec4b107ecfb9ea9e2830da3c8267bf2b28e4404f488e5b8 not found: ID does not exist"
Oct 30 19:45:25.595561 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.595501    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3bb27649-8419-414f-a8f0-23a1838ea701-kube-api-access-hmwfv\" (UniqueName: \"kubernetes.io/projected/3bb27649-8419-414f-a8f0-23a1838ea701-kube-api-access-hmwfv\") pod \"3bb27649-8419-414f-a8f0-23a1838ea701\" (UID: \"3bb27649-8419-414f-a8f0-23a1838ea701\") "
Oct 30 19:45:25.595776 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.595567    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3bb27649-8419-414f-a8f0-23a1838ea701-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bb27649-8419-414f-a8f0-23a1838ea701-metrics-certs\") pod \"3bb27649-8419-414f-a8f0-23a1838ea701\" (UID: \"3bb27649-8419-414f-a8f0-23a1838ea701\") "
Oct 30 19:45:25.595776 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.595724    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3bb27649-8419-414f-a8f0-23a1838ea701-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bb27649-8419-414f-a8f0-23a1838ea701-openshift-service-ca\") pod \"3bb27649-8419-414f-a8f0-23a1838ea701\" (UID: \"3bb27649-8419-414f-a8f0-23a1838ea701\") "
Oct 30 19:45:25.595932 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.595798    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3bb27649-8419-414f-a8f0-23a1838ea701-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bb27649-8419-414f-a8f0-23a1838ea701-maas-api-tls\") pod \"3bb27649-8419-414f-a8f0-23a1838ea701\" (UID: \"3bb27649-8419-414f-a8f0-23a1838ea701\") "
Oct 30 19:45:25.597080 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.597019    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bb27649-8419-414f-a8f0-23a1838ea701-openshift-service-ca" pod "3bb27649-8419-414f-a8f0-23a1838ea701" (UID: "3bb27649-8419-414f-a8f0-23a1838ea701"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:25.599420 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.598978    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bb27649-8419-414f-a8f0-23a1838ea701-kube-api-access-hmwfv" pod "3bb27649-8419-414f-a8f0-23a1838ea701" (UID: "3bb27649-8419-414f-a8f0-23a1838ea701"). InnerVolumeSpecName "kube-api-access-hmwfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:25.599710 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.599543    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bb27649-8419-414f-a8f0-23a1838ea701-metrics-certs" pod "3bb27649-8419-414f-a8f0-23a1838ea701" (UID: "3bb27649-8419-414f-a8f0-23a1838ea701"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:25.600053 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.599897    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bb27649-8419-414f-a8f0-23a1838ea701-maas-api-tls" pod "3bb27649-8419-414f-a8f0-23a1838ea701" (UID: "3bb27649-8419-414f-a8f0-23a1838ea701"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:25.697091 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.697058    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmwfv\" (UniqueName: \"kubernetes.io/projected/3bb27649-8419-414f-a8f0-23a1838ea701-kube-api-access-hmwfv\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:25.697091 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.697089    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bb27649-8419-414f-a8f0-23a1838ea701-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:25.697309 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.697103    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bb27649-8419-414f-a8f0-23a1838ea701-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:25.697309 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.697115    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bb27649-8419-414f-a8f0-23a1838ea701-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:25.949896 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.949861    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"]
Oct 30 19:45:25.953845 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:25.953822    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-5da1ae-9697c994b-pmjck"]
Oct 30 19:45:26.900407 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:26.900367    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:27.165864 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:27.165764    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:27.435173 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:27.435100    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3bb27649-8419-414f-a8f0-23a1838ea701" path="/var/lib/kubelet/pods/3bb27649-8419-414f-a8f0-23a1838ea701/volumes"
Oct 30 19:45:27.553135 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:27.553100    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:27.861385 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:27.861344    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:30.361093 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:30.361055    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw" podUID="420b7e12-f43e-45e0-be09-042d57313f5b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:45:30.361488 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:30.361055    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw" podUID="6211ab6d-429e-41e9-b0a0-3a1d76043165" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:45:32.236180 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:32.236147    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:34.605141 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:34.605103    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:34.862431 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:34.862348    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:35.076143 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:35.076106    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:35.544716 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:35.544667    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:37.363524 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:37.363487    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:38.202736 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:45:38.202702    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:45:40.137391 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.137328    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw"
Oct 30 19:45:40.241913 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.241890    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/420b7e12-f43e-45e0-be09-042d57313f5b-kube-api-access-4lh58\" (UniqueName: \"kubernetes.io/projected/420b7e12-f43e-45e0-be09-042d57313f5b-kube-api-access-4lh58\") pod \"420b7e12-f43e-45e0-be09-042d57313f5b\" (UID: \"420b7e12-f43e-45e0-be09-042d57313f5b\") "
Oct 30 19:45:40.242065 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.241934    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/420b7e12-f43e-45e0-be09-042d57313f5b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/420b7e12-f43e-45e0-be09-042d57313f5b-plugins-config-volume\") pod \"420b7e12-f43e-45e0-be09-042d57313f5b\" (UID: \"420b7e12-f43e-45e0-be09-042d57313f5b\") "
Oct 30 19:45:40.242375 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.242349    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/420b7e12-f43e-45e0-be09-042d57313f5b-plugins-config-volume" pod "420b7e12-f43e-45e0-be09-042d57313f5b" (UID: "420b7e12-f43e-45e0-be09-042d57313f5b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:40.244169 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.244146    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/420b7e12-f43e-45e0-be09-042d57313f5b-kube-api-access-4lh58" pod "420b7e12-f43e-45e0-be09-042d57313f5b" (UID: "420b7e12-f43e-45e0-be09-042d57313f5b"). InnerVolumeSpecName "kube-api-access-4lh58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:40.252907 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.252886    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw"
Oct 30 19:45:40.343551 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.343515    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6211ab6d-429e-41e9-b0a0-3a1d76043165-kube-api-access-d89bh\" (UniqueName: \"kubernetes.io/projected/6211ab6d-429e-41e9-b0a0-3a1d76043165-kube-api-access-d89bh\") pod \"6211ab6d-429e-41e9-b0a0-3a1d76043165\" (UID: \"6211ab6d-429e-41e9-b0a0-3a1d76043165\") "
Oct 30 19:45:40.343705 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.343609    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6211ab6d-429e-41e9-b0a0-3a1d76043165-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6211ab6d-429e-41e9-b0a0-3a1d76043165-plugins-config-volume\") pod \"6211ab6d-429e-41e9-b0a0-3a1d76043165\" (UID: \"6211ab6d-429e-41e9-b0a0-3a1d76043165\") "
Oct 30 19:45:40.343881 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.343869    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lh58\" (UniqueName: \"kubernetes.io/projected/420b7e12-f43e-45e0-be09-042d57313f5b-kube-api-access-4lh58\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:40.344005 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.343885    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/420b7e12-f43e-45e0-be09-042d57313f5b-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:40.344046 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.344019    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6211ab6d-429e-41e9-b0a0-3a1d76043165-plugins-config-volume" pod "6211ab6d-429e-41e9-b0a0-3a1d76043165" (UID: "6211ab6d-429e-41e9-b0a0-3a1d76043165"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:40.345828 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.345811    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6211ab6d-429e-41e9-b0a0-3a1d76043165-kube-api-access-d89bh" pod "6211ab6d-429e-41e9-b0a0-3a1d76043165" (UID: "6211ab6d-429e-41e9-b0a0-3a1d76043165"). InnerVolumeSpecName "kube-api-access-d89bh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:40.445242 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.445166    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6211ab6d-429e-41e9-b0a0-3a1d76043165-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:40.445242 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.445195    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d89bh\" (UniqueName: \"kubernetes.io/projected/6211ab6d-429e-41e9-b0a0-3a1d76043165-kube-api-access-d89bh\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:40.456449 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.456417    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"]
Oct 30 19:45:40.466697 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.466668    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"]
Oct 30 19:45:40.605095 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.605055    5820 generic.go:356] "Generic (PLEG): container finished" podID="420b7e12-f43e-45e0-be09-042d57313f5b" containerID="526bdff887834236f7b34d96a7d1f48eb2feea0d4f73cd8615863fe75a894919" exitCode=0
Oct 30 19:45:40.605095 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.605099    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw"
Oct 30 19:45:40.605326 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.605130    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw" event={"ID":"420b7e12-f43e-45e0-be09-042d57313f5b","Type":"ContainerDied","Data":"526bdff887834236f7b34d96a7d1f48eb2feea0d4f73cd8615863fe75a894919"}
Oct 30 19:45:40.605326 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.605189    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw" event={"ID":"420b7e12-f43e-45e0-be09-042d57313f5b","Type":"ContainerDied","Data":"e626d818a2b53e82d47cb316ff7a8b874d0066e31c9df84aebfcaf4425ce1a2d"}
Oct 30 19:45:40.605326 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.605214    5820 scope.go:122] "RemoveContainer" containerID="526bdff887834236f7b34d96a7d1f48eb2feea0d4f73cd8615863fe75a894919"
Oct 30 19:45:40.606617 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.606592    5820 generic.go:356] "Generic (PLEG): container finished" podID="6211ab6d-429e-41e9-b0a0-3a1d76043165" containerID="c9b3f6b82afb57b0eb1faa281b1e25ce512b28a3a21dd09a7d320d3bdd339887" exitCode=0
Oct 30 19:45:40.606742 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.606622    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw" event={"ID":"6211ab6d-429e-41e9-b0a0-3a1d76043165","Type":"ContainerDied","Data":"c9b3f6b82afb57b0eb1faa281b1e25ce512b28a3a21dd09a7d320d3bdd339887"}
Oct 30 19:45:40.606742 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.606650    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw"
Oct 30 19:45:40.606742 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.606698    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw" event={"ID":"6211ab6d-429e-41e9-b0a0-3a1d76043165","Type":"ContainerDied","Data":"fbcb096891ae94a47cc2eac6bde089945114bdf2b17a5d3741f4215ca53ec041"}
Oct 30 19:45:40.614753 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.614735    5820 scope.go:122] "RemoveContainer" containerID="526bdff887834236f7b34d96a7d1f48eb2feea0d4f73cd8615863fe75a894919"
Oct 30 19:45:40.615026 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:45:40.614999    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"526bdff887834236f7b34d96a7d1f48eb2feea0d4f73cd8615863fe75a894919\": container with ID starting with 526bdff887834236f7b34d96a7d1f48eb2feea0d4f73cd8615863fe75a894919 not found: ID does not exist" containerID="526bdff887834236f7b34d96a7d1f48eb2feea0d4f73cd8615863fe75a894919"
Oct 30 19:45:40.615109 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.615039    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"526bdff887834236f7b34d96a7d1f48eb2feea0d4f73cd8615863fe75a894919"} err="failed to get container status \"526bdff887834236f7b34d96a7d1f48eb2feea0d4f73cd8615863fe75a894919\": rpc error: code = NotFound desc = could not find container \"526bdff887834236f7b34d96a7d1f48eb2feea0d4f73cd8615863fe75a894919\": container with ID starting with 526bdff887834236f7b34d96a7d1f48eb2feea0d4f73cd8615863fe75a894919 not found: ID does not exist"
Oct 30 19:45:40.615109 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.615063    5820 scope.go:122] "RemoveContainer" containerID="c9b3f6b82afb57b0eb1faa281b1e25ce512b28a3a21dd09a7d320d3bdd339887"
Oct 30 19:45:40.623767 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.623749    5820 scope.go:122] "RemoveContainer" containerID="c9b3f6b82afb57b0eb1faa281b1e25ce512b28a3a21dd09a7d320d3bdd339887"
Oct 30 19:45:40.624051 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:45:40.624030    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9b3f6b82afb57b0eb1faa281b1e25ce512b28a3a21dd09a7d320d3bdd339887\": container with ID starting with c9b3f6b82afb57b0eb1faa281b1e25ce512b28a3a21dd09a7d320d3bdd339887 not found: ID does not exist" containerID="c9b3f6b82afb57b0eb1faa281b1e25ce512b28a3a21dd09a7d320d3bdd339887"
Oct 30 19:45:40.624126 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.624058    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9b3f6b82afb57b0eb1faa281b1e25ce512b28a3a21dd09a7d320d3bdd339887"} err="failed to get container status \"c9b3f6b82afb57b0eb1faa281b1e25ce512b28a3a21dd09a7d320d3bdd339887\": rpc error: code = NotFound desc = could not find container \"c9b3f6b82afb57b0eb1faa281b1e25ce512b28a3a21dd09a7d320d3bdd339887\": container with ID starting with c9b3f6b82afb57b0eb1faa281b1e25ce512b28a3a21dd09a7d320d3bdd339887 not found: ID does not exist"
Oct 30 19:45:40.669143 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.669120    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:40.806731 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.806699    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw"]
Oct 30 19:45:40.811092 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.811068    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-5da1ae-599786b944-qs8jw"]
Oct 30 19:45:40.920250 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.920219    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw"]
Oct 30 19:45:40.925315 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:40.925292    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-5da1ae-7689968659-w84kw"]
Oct 30 19:45:41.434824 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:41.434751    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="420b7e12-f43e-45e0-be09-042d57313f5b" path="/var/lib/kubelet/pods/420b7e12-f43e-45e0-be09-042d57313f5b/volumes"
Oct 30 19:45:41.435201 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:41.435172    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6211ab6d-429e-41e9-b0a0-3a1d76043165" path="/var/lib/kubelet/pods/6211ab6d-429e-41e9-b0a0-3a1d76043165/volumes"
Oct 30 19:45:42.228757 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:42.228722    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:47.473619 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:47.473582    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz"]
Oct 30 19:45:47.872779 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:47.872739    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:48.147082 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:48.146994    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:52.474797 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:52.474743    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz" podUID="6579c071-4113-46cb-a9bd-c5d128f57a5b" containerName="payload-processing" containerID="cri-o://1fb5ef5122200d4624ff7683bfd71560789cf521faf0e238b19faf8d479ca18c" gracePeriod=25
Oct 30 19:45:53.852053 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:53.852009    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:54.081399 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:54.081355    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:55.477118 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.477058    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b" podUID="1cae2237-0916-415a-9059-b94e0d950b0a" containerName="main" containerID="cri-o://423139a8afde64a3b1ab746212e7a52fa160319f26adb1d696cb35b568e8486f" gracePeriod=45
Oct 30 19:45:55.489343 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.489310    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw" podUID="1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" containerName="main" containerID="cri-o://3fdd16eb7b7436ad0fb2da1ef90eae602c3cd4198c0cdeb347ccfaad0b519541" gracePeriod=45
Oct 30 19:45:55.676369 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.676321    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw" podUID="1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": dial tcp 10.134.0.63:8000: connect: connection refused"
Oct 30 19:45:55.681164 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.681134    5820 generic.go:356] "Generic (PLEG): container finished" podID="1cae2237-0916-415a-9059-b94e0d950b0a" containerID="423139a8afde64a3b1ab746212e7a52fa160319f26adb1d696cb35b568e8486f" exitCode=0
Oct 30 19:45:55.681328 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.681198    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b" event={"ID":"1cae2237-0916-415a-9059-b94e0d950b0a","Type":"ContainerDied","Data":"423139a8afde64a3b1ab746212e7a52fa160319f26adb1d696cb35b568e8486f"}
Oct 30 19:45:55.682913 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.682889    5820 generic.go:356] "Generic (PLEG): container finished" podID="1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" containerID="3fdd16eb7b7436ad0fb2da1ef90eae602c3cd4198c0cdeb347ccfaad0b519541" exitCode=0
Oct 30 19:45:55.683100 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.682970    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw" event={"ID":"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0","Type":"ContainerDied","Data":"3fdd16eb7b7436ad0fb2da1ef90eae602c3cd4198c0cdeb347ccfaad0b519541"}
Oct 30 19:45:55.734420 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.734363    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 19:45:55.765489 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.765466    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 19:45:55.787615 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.787587    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-model-cache\") pod \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") "
Oct 30 19:45:55.787737 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.787624    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-home\" (UniqueName: \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-home\") pod \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") "
Oct 30 19:45:55.787737 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.787677    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-dshm\") pod \"1cae2237-0916-415a-9059-b94e0d950b0a\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") "
Oct 30 19:45:55.787737 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.787715    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-home\" (UniqueName: \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-home\") pod \"1cae2237-0916-415a-9059-b94e0d950b0a\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") "
Oct 30 19:45:55.787916 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.787747    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1cae2237-0916-415a-9059-b94e0d950b0a-kube-api-access-mj84j\" (UniqueName: \"kubernetes.io/projected/1cae2237-0916-415a-9059-b94e0d950b0a-kube-api-access-mj84j\") pod \"1cae2237-0916-415a-9059-b94e0d950b0a\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") "
Oct 30 19:45:55.787916 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.787800    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1cae2237-0916-415a-9059-b94e0d950b0a-tls-certs\" (UniqueName: \"kubernetes.io/secret/1cae2237-0916-415a-9059-b94e0d950b0a-tls-certs\") pod \"1cae2237-0916-415a-9059-b94e0d950b0a\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") "
Oct 30 19:45:55.787916 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.787860    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-dshm\") pod \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") "
Oct 30 19:45:55.787916 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.787863    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-model-cache" pod "1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" (UID: "1d33ec3b-bb2f-428f-86dd-42ad3c7009d0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:55.788122 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.787979    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-home" pod "1cae2237-0916-415a-9059-b94e0d950b0a" (UID: "1cae2237-0916-415a-9059-b94e0d950b0a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:55.788122 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.787994    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-home" pod "1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" (UID: "1d33ec3b-bb2f-428f-86dd-42ad3c7009d0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:55.788218 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.788000    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-kube-api-access-cqf5h\" (UniqueName: \"kubernetes.io/projected/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-kube-api-access-cqf5h\") pod \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") "
Oct 30 19:45:55.788281 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.788259    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-model-cache\") pod \"1cae2237-0916-415a-9059-b94e0d950b0a\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") "
Oct 30 19:45:55.788330 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.788310    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-tmp-dir\") pod \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") "
Oct 30 19:45:55.788403 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.788355    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-tmp-dir\") pod \"1cae2237-0916-415a-9059-b94e0d950b0a\" (UID: \"1cae2237-0916-415a-9059-b94e0d950b0a\") "
Oct 30 19:45:55.788403 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.788389    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-tls-certs\" (UniqueName: \"kubernetes.io/secret/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-tls-certs\") pod \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\" (UID: \"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0\") "
Oct 30 19:45:55.788608 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.788587    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-tmp-dir" pod "1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" (UID: "1d33ec3b-bb2f-428f-86dd-42ad3c7009d0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:55.788763 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.788742    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-tmp-dir" pod "1cae2237-0916-415a-9059-b94e0d950b0a" (UID: "1cae2237-0916-415a-9059-b94e0d950b0a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:55.789214 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.789038    5820 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-model-cache\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.789214 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.789112    5820 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-home\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.789214 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.789129    5820 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-home\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.789214 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.789179    5820 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-tmp-dir\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.789214 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.789193    5820 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-tmp-dir\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.789561 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.789189    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-model-cache" pod "1cae2237-0916-415a-9059-b94e0d950b0a" (UID: "1cae2237-0916-415a-9059-b94e0d950b0a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:55.790738 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.790707    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-dshm" pod "1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" (UID: "1d33ec3b-bb2f-428f-86dd-42ad3c7009d0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:55.790845 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.790812    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-tls-certs" pod "1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" (UID: "1d33ec3b-bb2f-428f-86dd-42ad3c7009d0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:55.790845 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.790823    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-dshm" pod "1cae2237-0916-415a-9059-b94e0d950b0a" (UID: "1cae2237-0916-415a-9059-b94e0d950b0a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:55.790929 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.790859    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-kube-api-access-cqf5h" pod "1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" (UID: "1d33ec3b-bb2f-428f-86dd-42ad3c7009d0"). InnerVolumeSpecName "kube-api-access-cqf5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:55.791024 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.791002    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cae2237-0916-415a-9059-b94e0d950b0a-tls-certs" pod "1cae2237-0916-415a-9059-b94e0d950b0a" (UID: "1cae2237-0916-415a-9059-b94e0d950b0a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:55.791879 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.791863    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cae2237-0916-415a-9059-b94e0d950b0a-kube-api-access-mj84j" pod "1cae2237-0916-415a-9059-b94e0d950b0a" (UID: "1cae2237-0916-415a-9059-b94e0d950b0a"). InnerVolumeSpecName "kube-api-access-mj84j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:55.890062 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.890030    5820 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-model-cache\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.890062 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.890058    5820 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-tls-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.890239 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.890071    5820 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cae2237-0916-415a-9059-b94e0d950b0a-dshm\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.890239 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.890081    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mj84j\" (UniqueName: \"kubernetes.io/projected/1cae2237-0916-415a-9059-b94e0d950b0a-kube-api-access-mj84j\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.890239 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.890091    5820 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cae2237-0916-415a-9059-b94e0d950b0a-tls-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.890239 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.890099    5820 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-dshm\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.890239 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:55.890107    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqf5h\" (UniqueName: \"kubernetes.io/projected/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0-kube-api-access-cqf5h\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:56.116611 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:56.116574    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz" podUID="6579c071-4113-46cb-a9bd-c5d128f57a5b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:45:56.689823 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:56.689794    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"
Oct 30 19:45:56.690351 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:56.689840    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw" event={"ID":"1d33ec3b-bb2f-428f-86dd-42ad3c7009d0","Type":"ContainerDied","Data":"bf3b18f4d10a7c30ba0066a793908586f54f065993818db58170ca3a8e55a35c"}
Oct 30 19:45:56.690351 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:56.689903    5820 scope.go:122] "RemoveContainer" containerID="3fdd16eb7b7436ad0fb2da1ef90eae602c3cd4198c0cdeb347ccfaad0b519541"
Oct 30 19:45:56.691301 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:56.691277    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b" event={"ID":"1cae2237-0916-415a-9059-b94e0d950b0a","Type":"ContainerDied","Data":"639bcd2e050e734adc79c03e0a5ae69fb4e408eb8b914e9fa68a60d5d68b20b2"}
Oct 30 19:45:56.691406 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:56.691291    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"
Oct 30 19:45:56.705760 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:56.705733    5820 scope.go:122] "RemoveContainer" containerID="423139a8afde64a3b1ab746212e7a52fa160319f26adb1d696cb35b568e8486f"
Oct 30 19:45:56.854529 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:56.854501    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"]
Oct 30 19:45:56.903935 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:56.903894    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/simulator-w4-e7929d-kserve-6fc5dc86f7-7kbrw"]
Oct 30 19:45:56.991779 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:56.991753    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"]
Oct 30 19:45:56.998007 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:56.997985    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/premium-w4-e7929d-kserve-8644b74cbd-nwv2b"]
Oct 30 19:45:57.435491 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:57.435460    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1cae2237-0916-415a-9059-b94e0d950b0a" path="/var/lib/kubelet/pods/1cae2237-0916-415a-9059-b94e0d950b0a/volumes"
Oct 30 19:45:57.435858 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:45:57.435845    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" path="/var/lib/kubelet/pods/1d33ec3b-bb2f-428f-86dd-42ad3c7009d0/volumes"
Oct 30 19:46:01.222804 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.222770    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"]
Oct 30 19:46:01.223252 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.223009    5820 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4" podUID="27cbbf2c-1014-49b5-86b8-de8041a0b268" containerName="maas-api" containerID="cri-o://eeb054b9352526886aca6d37dea8aab52f4a8ea0a3ac5ca21618ce8495dccf88" gracePeriod=30
Oct 30 19:46:01.372026 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.371991    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:01.481264 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.481210    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"
Oct 30 19:46:01.539700 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.539673    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/27cbbf2c-1014-49b5-86b8-de8041a0b268-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27cbbf2c-1014-49b5-86b8-de8041a0b268-openshift-service-ca\") pod \"27cbbf2c-1014-49b5-86b8-de8041a0b268\" (UID: \"27cbbf2c-1014-49b5-86b8-de8041a0b268\") "
Oct 30 19:46:01.539828 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.539717    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/27cbbf2c-1014-49b5-86b8-de8041a0b268-kube-api-access-bf8hz\" (UniqueName: \"kubernetes.io/projected/27cbbf2c-1014-49b5-86b8-de8041a0b268-kube-api-access-bf8hz\") pod \"27cbbf2c-1014-49b5-86b8-de8041a0b268\" (UID: \"27cbbf2c-1014-49b5-86b8-de8041a0b268\") "
Oct 30 19:46:01.539828 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.539754    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/27cbbf2c-1014-49b5-86b8-de8041a0b268-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27cbbf2c-1014-49b5-86b8-de8041a0b268-maas-api-tls\") pod \"27cbbf2c-1014-49b5-86b8-de8041a0b268\" (UID: \"27cbbf2c-1014-49b5-86b8-de8041a0b268\") "
Oct 30 19:46:01.539828 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.539781    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/27cbbf2c-1014-49b5-86b8-de8041a0b268-metrics-certs\" (UniqueName: \"kubernetes.io/secret/27cbbf2c-1014-49b5-86b8-de8041a0b268-metrics-certs\") pod \"27cbbf2c-1014-49b5-86b8-de8041a0b268\" (UID: \"27cbbf2c-1014-49b5-86b8-de8041a0b268\") "
Oct 30 19:46:01.540128 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.540104    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27cbbf2c-1014-49b5-86b8-de8041a0b268-openshift-service-ca" pod "27cbbf2c-1014-49b5-86b8-de8041a0b268" (UID: "27cbbf2c-1014-49b5-86b8-de8041a0b268"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:01.540827 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.540615    5820 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27cbbf2c-1014-49b5-86b8-de8041a0b268-openshift-service-ca\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:46:01.542241 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.542215    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27cbbf2c-1014-49b5-86b8-de8041a0b268-metrics-certs" pod "27cbbf2c-1014-49b5-86b8-de8041a0b268" (UID: "27cbbf2c-1014-49b5-86b8-de8041a0b268"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:01.542352 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.542253    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27cbbf2c-1014-49b5-86b8-de8041a0b268-maas-api-tls" pod "27cbbf2c-1014-49b5-86b8-de8041a0b268" (UID: "27cbbf2c-1014-49b5-86b8-de8041a0b268"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:01.542352 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.542271    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27cbbf2c-1014-49b5-86b8-de8041a0b268-kube-api-access-bf8hz" pod "27cbbf2c-1014-49b5-86b8-de8041a0b268" (UID: "27cbbf2c-1014-49b5-86b8-de8041a0b268"). InnerVolumeSpecName "kube-api-access-bf8hz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:01.641927 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.641894    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bf8hz\" (UniqueName: \"kubernetes.io/projected/27cbbf2c-1014-49b5-86b8-de8041a0b268-kube-api-access-bf8hz\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:46:01.641927 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.641928    5820 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27cbbf2c-1014-49b5-86b8-de8041a0b268-maas-api-tls\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:46:01.642224 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.641959    5820 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27cbbf2c-1014-49b5-86b8-de8041a0b268-metrics-certs\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:46:01.722528 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.722495    5820 generic.go:356] "Generic (PLEG): container finished" podID="27cbbf2c-1014-49b5-86b8-de8041a0b268" containerID="eeb054b9352526886aca6d37dea8aab52f4a8ea0a3ac5ca21618ce8495dccf88" exitCode=0
Oct 30 19:46:01.722699 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.722554    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4" event={"ID":"27cbbf2c-1014-49b5-86b8-de8041a0b268","Type":"ContainerDied","Data":"eeb054b9352526886aca6d37dea8aab52f4a8ea0a3ac5ca21618ce8495dccf88"}
Oct 30 19:46:01.722699 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.722560    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"
Oct 30 19:46:01.722699 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.722597    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4" event={"ID":"27cbbf2c-1014-49b5-86b8-de8041a0b268","Type":"ContainerDied","Data":"1a036e5d35121683e26e65289535c3a783610ad187fa5e43d802fc4f305f5420"}
Oct 30 19:46:01.722699 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.722624    5820 scope.go:122] "RemoveContainer" containerID="eeb054b9352526886aca6d37dea8aab52f4a8ea0a3ac5ca21618ce8495dccf88"
Oct 30 19:46:01.731998 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.731981    5820 scope.go:122] "RemoveContainer" containerID="eeb054b9352526886aca6d37dea8aab52f4a8ea0a3ac5ca21618ce8495dccf88"
Oct 30 19:46:01.732225 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:46:01.732207    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eeb054b9352526886aca6d37dea8aab52f4a8ea0a3ac5ca21618ce8495dccf88\": container with ID starting with eeb054b9352526886aca6d37dea8aab52f4a8ea0a3ac5ca21618ce8495dccf88 not found: ID does not exist" containerID="eeb054b9352526886aca6d37dea8aab52f4a8ea0a3ac5ca21618ce8495dccf88"
Oct 30 19:46:01.732278 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.732233    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeb054b9352526886aca6d37dea8aab52f4a8ea0a3ac5ca21618ce8495dccf88"} err="failed to get container status \"eeb054b9352526886aca6d37dea8aab52f4a8ea0a3ac5ca21618ce8495dccf88\": rpc error: code = NotFound desc = could not find container \"eeb054b9352526886aca6d37dea8aab52f4a8ea0a3ac5ca21618ce8495dccf88\": container with ID starting with eeb054b9352526886aca6d37dea8aab52f4a8ea0a3ac5ca21618ce8495dccf88 not found: ID does not exist"
Oct 30 19:46:01.887780 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.887748    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"]
Oct 30 19:46:01.889849 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:01.889827    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e7929d-76b77b8847-d75t4"]
Oct 30 19:46:02.890877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:02.890841    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:03.174089 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:03.173996    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:03.436158 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:03.436084    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="27cbbf2c-1014-49b5-86b8-de8041a0b268" path="/var/lib/kubelet/pods/27cbbf2c-1014-49b5-86b8-de8041a0b268/volumes"
Oct 30 19:46:03.835017 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:03.834981    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:06.116842 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:06.116806    5820 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz" podUID="6579c071-4113-46cb-a9bd-c5d128f57a5b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:46:08.202896 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:46:08.202862    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:46:10.726071 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:10.726035    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:10.948537 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:10.948505    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:11.135543 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:11.135505    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:12.725377 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:12.725355    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz"
Oct 30 19:46:12.781390 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:12.781304    5820 generic.go:356] "Generic (PLEG): container finished" podID="6579c071-4113-46cb-a9bd-c5d128f57a5b" containerID="1fb5ef5122200d4624ff7683bfd71560789cf521faf0e238b19faf8d479ca18c" exitCode=0
Oct 30 19:46:12.781390 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:12.781351    5820 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz"
Oct 30 19:46:12.781390 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:12.781359    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz" event={"ID":"6579c071-4113-46cb-a9bd-c5d128f57a5b","Type":"ContainerDied","Data":"1fb5ef5122200d4624ff7683bfd71560789cf521faf0e238b19faf8d479ca18c"}
Oct 30 19:46:12.781655 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:12.781398    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz" event={"ID":"6579c071-4113-46cb-a9bd-c5d128f57a5b","Type":"ContainerDied","Data":"2a38a3d40ef9f3d916972ba61a845399c9e92da7cc93988315600bf9300cb4a3"}
Oct 30 19:46:12.781655 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:12.781422    5820 scope.go:122] "RemoveContainer" containerID="1fb5ef5122200d4624ff7683bfd71560789cf521faf0e238b19faf8d479ca18c"
Oct 30 19:46:12.790031 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:12.790010    5820 scope.go:122] "RemoveContainer" containerID="1fb5ef5122200d4624ff7683bfd71560789cf521faf0e238b19faf8d479ca18c"
Oct 30 19:46:12.790291 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:46:12.790272    5820 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fb5ef5122200d4624ff7683bfd71560789cf521faf0e238b19faf8d479ca18c\": container with ID starting with 1fb5ef5122200d4624ff7683bfd71560789cf521faf0e238b19faf8d479ca18c not found: ID does not exist" containerID="1fb5ef5122200d4624ff7683bfd71560789cf521faf0e238b19faf8d479ca18c"
Oct 30 19:46:12.790345 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:12.790304    5820 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fb5ef5122200d4624ff7683bfd71560789cf521faf0e238b19faf8d479ca18c"} err="failed to get container status \"1fb5ef5122200d4624ff7683bfd71560789cf521faf0e238b19faf8d479ca18c\": rpc error: code = NotFound desc = could not find container \"1fb5ef5122200d4624ff7683bfd71560789cf521faf0e238b19faf8d479ca18c\": container with ID starting with 1fb5ef5122200d4624ff7683bfd71560789cf521faf0e238b19faf8d479ca18c not found: ID does not exist"
Oct 30 19:46:12.845446 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:12.845419    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6579c071-4113-46cb-a9bd-c5d128f57a5b-kube-api-access-hhfct\" (UniqueName: \"kubernetes.io/projected/6579c071-4113-46cb-a9bd-c5d128f57a5b-kube-api-access-hhfct\") pod \"6579c071-4113-46cb-a9bd-c5d128f57a5b\" (UID: \"6579c071-4113-46cb-a9bd-c5d128f57a5b\") "
Oct 30 19:46:12.845620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:12.845486    5820 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6579c071-4113-46cb-a9bd-c5d128f57a5b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6579c071-4113-46cb-a9bd-c5d128f57a5b-plugins-config-volume\") pod \"6579c071-4113-46cb-a9bd-c5d128f57a5b\" (UID: \"6579c071-4113-46cb-a9bd-c5d128f57a5b\") "
Oct 30 19:46:12.845866 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:12.845844    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6579c071-4113-46cb-a9bd-c5d128f57a5b-plugins-config-volume" pod "6579c071-4113-46cb-a9bd-c5d128f57a5b" (UID: "6579c071-4113-46cb-a9bd-c5d128f57a5b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:12.847795 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:12.847756    5820 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6579c071-4113-46cb-a9bd-c5d128f57a5b-kube-api-access-hhfct" pod "6579c071-4113-46cb-a9bd-c5d128f57a5b" (UID: "6579c071-4113-46cb-a9bd-c5d128f57a5b"). InnerVolumeSpecName "kube-api-access-hhfct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:12.946583 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:12.946549    5820 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhfct\" (UniqueName: \"kubernetes.io/projected/6579c071-4113-46cb-a9bd-c5d128f57a5b-kube-api-access-hhfct\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:46:12.946583 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:12.946577    5820 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6579c071-4113-46cb-a9bd-c5d128f57a5b-plugins-config-volume\") on node \"ip-10-0-133-80.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:46:13.204827 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:13.204800    5820 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz"]
Oct 30 19:46:13.211026 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:13.210993    5820 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-e7929d-79b58b5686-lnwbz"]
Oct 30 19:46:13.435478 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:13.435404    5820 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6579c071-4113-46cb-a9bd-c5d128f57a5b" path="/var/lib/kubelet/pods/6579c071-4113-46cb-a9bd-c5d128f57a5b/volumes"
Oct 30 19:46:15.965512 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:15.965476    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-84cc5ff9cd-7vt6f_f3c7887d-5d2f-47a9-8808-91251d470724/manager/0.log"
Oct 30 19:46:16.462516 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:16.462470    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-dtgl9_76f5d060-4e95-48d3-99e3-815bac778442/manager/0.log"
Oct 30 19:46:17.322634 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:17.322604    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:17.547354 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:17.547320    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:17.648176 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:17.648102    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-vbdd5_2452b1f3-519b-4e1d-ad93-16c233d34895/server/0.log"
Oct 30 19:46:17.892558 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:17.892525    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-jbdx9_48168bb8-0b54-4f25-9139-0ddc3f7950d6/manager/0.log"
Oct 30 19:46:18.162064 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:18.162029    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-mcgqw_d0e5ae11-c274-4e34-a6fe-ac8df2700247/proxy/0.log"
Oct 30 19:46:19.071365 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:19.071330    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:19.271513 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:19.271483    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:25.087079 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:25.087027    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:29.723045 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:29.723010    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-84cc5ff9cd-7vt6f_f3c7887d-5d2f-47a9-8808-91251d470724/manager/0.log"
Oct 30 19:46:30.231901 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:30.231872    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-dtgl9_76f5d060-4e95-48d3-99e3-815bac778442/manager/0.log"
Oct 30 19:46:31.343221 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:31.343190    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-vbdd5_2452b1f3-519b-4e1d-ad93-16c233d34895/server/0.log"
Oct 30 19:46:31.580877 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:31.580849    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-jbdx9_48168bb8-0b54-4f25-9139-0ddc3f7950d6/manager/0.log"
Oct 30 19:46:31.850398 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:31.850364    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-mcgqw_d0e5ae11-c274-4e34-a6fe-ac8df2700247/proxy/0.log"
Oct 30 19:46:32.891698 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:32.891664    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-776f8954dc-s9lws_60db3aee-1f1d-404f-abfb-e6f2ea221b3e/maas-api/0.log"
Oct 30 19:46:34.286095 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:34.286061    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-792a70a1-l4ncr_52e18e09-877a-4bd7-971d-90d489e5102a/revoke-keys/0.log"
Oct 30 19:46:35.207402 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:35.207375    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-j57nn_4bab49b9-aab3-4d20-abc5-ed06cd6964fa/postgres/0.log"
Oct 30 19:46:38.207625 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:46:38.207587    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:46:39.923454 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:39.923412    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-94chw_75209be8-a14e-40e9-9db9-d0a276c958f2/discovery/0.log"
Oct 30 19:46:40.403157 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:40.403126    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9468cdd5d-zg5gs_4de42b07-6abb-419a-99dc-89b226a2fa4e/kube-auth-proxy/0.log"
Oct 30 19:46:41.145408 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:41.145375    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-qmbs6_3e933bb3-b9ba-4dff-841e-831b32f20537/payload-pre-processing/0.log"
Oct 30 19:46:41.369906 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:41.369881    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-7kxzm_88b8cb4c-656b-44e4-8632-0acff65c58ad/payload-processing/0.log"
Oct 30 19:46:42.412961 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:42.412923    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-v42d8_aa596309-c2e8-4ba3-bf40-02fedf140bb5/main/0.log"
Oct 30 19:46:42.685836 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:42.685752    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-bjq4x_e273ced8-b9e7-479d-b9d5-33101d01e532/main/0.log"
Oct 30 19:46:43.639878 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:43.639847    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-k5crh_971c8953-1bc9-4276-9767-5a99d6a215ba/main/0.log"
Oct 30 19:46:43.877259 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:43.877231    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-n8j7t_a765d287-6e9e-408e-91b6-5a32ab049574/main/0.log"
Oct 30 19:46:54.665009 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.664921    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:54.715578 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.715544    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-z4nbd/must-gather-g4sgq"]
Oct 30 19:46:54.716035 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716021    5820 cpu_manager.go:414] "containerMap: removing container" podUID="27cbbf2c-1014-49b5-86b8-de8041a0b268" containerName="maas-api"
Oct 30 19:46:54.716093 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716037    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="27cbbf2c-1014-49b5-86b8-de8041a0b268" containerName="maas-api"
Oct 30 19:46:54.716093 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716054    5820 cpu_manager.go:414] "containerMap: removing container" podUID="1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" containerName="main"
Oct 30 19:46:54.716093 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716059    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" containerName="main"
Oct 30 19:46:54.716093 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716071    5820 cpu_manager.go:414] "containerMap: removing container" podUID="420b7e12-f43e-45e0-be09-042d57313f5b" containerName="payload-processing"
Oct 30 19:46:54.716093 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716078    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="420b7e12-f43e-45e0-be09-042d57313f5b" containerName="payload-processing"
Oct 30 19:46:54.716093 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716085    5820 cpu_manager.go:414] "containerMap: removing container" podUID="e5771fcc-1572-4603-afc0-90fd231b731f" containerName="payload-processing"
Oct 30 19:46:54.716093 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716090    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5771fcc-1572-4603-afc0-90fd231b731f" containerName="payload-processing"
Oct 30 19:46:54.716322 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716109    5820 cpu_manager.go:414] "containerMap: removing container" podUID="6579c071-4113-46cb-a9bd-c5d128f57a5b" containerName="payload-processing"
Oct 30 19:46:54.716322 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716114    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6579c071-4113-46cb-a9bd-c5d128f57a5b" containerName="payload-processing"
Oct 30 19:46:54.716322 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716122    5820 cpu_manager.go:414] "containerMap: removing container" podUID="3bb27649-8419-414f-a8f0-23a1838ea701" containerName="maas-api"
Oct 30 19:46:54.716322 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716126    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3bb27649-8419-414f-a8f0-23a1838ea701" containerName="maas-api"
Oct 30 19:46:54.716322 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716142    5820 cpu_manager.go:414] "containerMap: removing container" podUID="6211ab6d-429e-41e9-b0a0-3a1d76043165" containerName="payload-pre-processing"
Oct 30 19:46:54.716322 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716148    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6211ab6d-429e-41e9-b0a0-3a1d76043165" containerName="payload-pre-processing"
Oct 30 19:46:54.716322 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716165    5820 cpu_manager.go:414] "containerMap: removing container" podUID="1cae2237-0916-415a-9059-b94e0d950b0a" containerName="main"
Oct 30 19:46:54.716322 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716170    5820 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1cae2237-0916-415a-9059-b94e0d950b0a" containerName="main"
Oct 30 19:46:54.716322 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716237    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="1d33ec3b-bb2f-428f-86dd-42ad3c7009d0" containerName="main"
Oct 30 19:46:54.716322 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716245    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="420b7e12-f43e-45e0-be09-042d57313f5b" containerName="payload-processing"
Oct 30 19:46:54.716322 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716251    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="e5771fcc-1572-4603-afc0-90fd231b731f" containerName="payload-processing"
Oct 30 19:46:54.716322 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716260    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="3bb27649-8419-414f-a8f0-23a1838ea701" containerName="maas-api"
Oct 30 19:46:54.716322 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716265    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="6579c071-4113-46cb-a9bd-c5d128f57a5b" containerName="payload-processing"
Oct 30 19:46:54.716322 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716271    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="6211ab6d-429e-41e9-b0a0-3a1d76043165" containerName="payload-pre-processing"
Oct 30 19:46:54.716322 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716278    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="1cae2237-0916-415a-9059-b94e0d950b0a" containerName="main"
Oct 30 19:46:54.716322 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.716285    5820 memory_manager.go:365] "RemoveStaleState removing state" podUID="27cbbf2c-1014-49b5-86b8-de8041a0b268" containerName="maas-api"
Oct 30 19:46:54.719490 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.719469    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z4nbd/must-gather-g4sgq"
Oct 30 19:46:54.778471 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.778443    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z4nbd/must-gather-g4sgq"]
Oct 30 19:46:54.820258 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.820228    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2c54\" (UniqueName: \"kubernetes.io/projected/4998e2b2-107d-4063-ada3-7814f2d5c211-kube-api-access-d2c54\") pod \"must-gather-g4sgq\" (UID: \"4998e2b2-107d-4063-ada3-7814f2d5c211\") " pod="openshift-must-gather-z4nbd/must-gather-g4sgq"
Oct 30 19:46:54.820420 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.820275    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4998e2b2-107d-4063-ada3-7814f2d5c211-must-gather-output\") pod \"must-gather-g4sgq\" (UID: \"4998e2b2-107d-4063-ada3-7814f2d5c211\") " pod="openshift-must-gather-z4nbd/must-gather-g4sgq"
Oct 30 19:46:54.825477 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.825454    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z4nbd\"/\"openshift-service-ca.crt\""
Oct 30 19:46:54.826668 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.826651    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-z4nbd\"/\"default-dockercfg-qszzv\""
Oct 30 19:46:54.826745 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.826658    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z4nbd\"/\"kube-root-ca.crt\""
Oct 30 19:46:54.871885 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.871849    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:54.921238 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.921173    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2c54\" (UniqueName: \"kubernetes.io/projected/4998e2b2-107d-4063-ada3-7814f2d5c211-kube-api-access-d2c54\") pod \"must-gather-g4sgq\" (UID: \"4998e2b2-107d-4063-ada3-7814f2d5c211\") " pod="openshift-must-gather-z4nbd/must-gather-g4sgq"
Oct 30 19:46:54.921238 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.921225    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4998e2b2-107d-4063-ada3-7814f2d5c211-must-gather-output\") pod \"must-gather-g4sgq\" (UID: \"4998e2b2-107d-4063-ada3-7814f2d5c211\") " pod="openshift-must-gather-z4nbd/must-gather-g4sgq"
Oct 30 19:46:54.921555 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.921538    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4998e2b2-107d-4063-ada3-7814f2d5c211-must-gather-output\") pod \"must-gather-g4sgq\" (UID: \"4998e2b2-107d-4063-ada3-7814f2d5c211\") " pod="openshift-must-gather-z4nbd/must-gather-g4sgq"
Oct 30 19:46:54.981828 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:54.981804    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2c54\" (UniqueName: \"kubernetes.io/projected/4998e2b2-107d-4063-ada3-7814f2d5c211-kube-api-access-d2c54\") pod \"must-gather-g4sgq\" (UID: \"4998e2b2-107d-4063-ada3-7814f2d5c211\") " pod="openshift-must-gather-z4nbd/must-gather-g4sgq"
Oct 30 19:46:55.043211 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:55.043188    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z4nbd/must-gather-g4sgq"
Oct 30 19:46:55.420244 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:55.420048    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z4nbd/must-gather-g4sgq"]
Oct 30 19:46:55.422690 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:46:55.422666    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4998e2b2_107d_4063_ada3_7814f2d5c211.slice/crio-3e84065bd3483d44767ed00ffa1c560433aa433d96eb447cba7503da89878276 WatchSource:0}: Error finding container 3e84065bd3483d44767ed00ffa1c560433aa433d96eb447cba7503da89878276: Status 404 returned error can't find the container with id 3e84065bd3483d44767ed00ffa1c560433aa433d96eb447cba7503da89878276
Oct 30 19:46:55.983836 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:46:55.983804    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z4nbd/must-gather-g4sgq" event={"ID":"4998e2b2-107d-4063-ada3-7814f2d5c211","Type":"ContainerStarted","Data":"3e84065bd3483d44767ed00ffa1c560433aa433d96eb447cba7503da89878276"}
Oct 30 19:47:03.026263 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:03.026216    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z4nbd/must-gather-g4sgq" event={"ID":"4998e2b2-107d-4063-ada3-7814f2d5c211","Type":"ContainerStarted","Data":"44874bd5b58b4b6d70253f0445b338ae445f145e346c8514fca923360b6518ab"}
Oct 30 19:47:03.026671 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:03.026272    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z4nbd/must-gather-g4sgq" event={"ID":"4998e2b2-107d-4063-ada3-7814f2d5c211","Type":"ContainerStarted","Data":"bdc24d1bf445779ad83ff16c36afabae8fadbeee3da83f6110780eafbfe73459"}
Oct 30 19:47:03.142531 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:03.142466    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-z4nbd/must-gather-g4sgq" podStartSLOduration=2.302762451 podStartE2EDuration="9.142447616s" totalImagesPullingTime="6.839685165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:46:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:03.140413974 +0000 UTC m=+4343.852162675" watchObservedRunningTime="2026-10-30 19:47:03.142447616 +0000 UTC m=+4343.854196315"
Oct 30 19:47:05.285084 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:05.285048    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gfs79_6300215e-2302-47ed-bf47-05a165277c08/global-pull-secret-syncer/0.log"
Oct 30 19:47:06.678611 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:06.678577    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-84cc5ff9cd-7vt6f_f3c7887d-5d2f-47a9-8808-91251d470724/manager/0.log"
Oct 30 19:47:07.146429 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:07.146392    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-dtgl9_76f5d060-4e95-48d3-99e3-815bac778442/manager/0.log"
Oct 30 19:47:07.436161 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:07.436085    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log"
Oct 30 19:47:08.204687 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:47:08.204649    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:47:08.264578 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:08.264541    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-84cc5ff9cd-7vt6f_f3c7887d-5d2f-47a9-8808-91251d470724/manager/0.log"
Oct 30 19:47:08.504151 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:08.504081    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-vbdd5_2452b1f3-519b-4e1d-ad93-16c233d34895/server/0.log"
Oct 30 19:47:08.763080 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:08.762985    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-dtgl9_76f5d060-4e95-48d3-99e3-815bac778442/manager/0.log"
Oct 30 19:47:09.092362 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:09.092331    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-80.us-west-1.compute.internal_ff8defe0c0f1527a0efd6540e2b0bdc3/haproxy/0.log"
Oct 30 19:47:09.892005 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:09.891939    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-vbdd5_2452b1f3-519b-4e1d-ad93-16c233d34895/server/0.log"
Oct 30 19:47:09.999223 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:09.999190    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-jbdx9_48168bb8-0b54-4f25-9139-0ddc3f7950d6/manager/0.log"
Oct 30 19:47:10.123455 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:10.123425    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-mcgqw_d0e5ae11-c274-4e34-a6fe-ac8df2700247/proxy/0.log"
Oct 30 19:47:11.352637 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:11.352582    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-jbdx9_48168bb8-0b54-4f25-9139-0ddc3f7950d6/manager/0.log"
Oct 30 19:47:11.356645 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:11.356615    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:11.473727 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:11.473697    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-mcgqw_d0e5ae11-c274-4e34-a6fe-ac8df2700247/proxy/0.log"
Oct 30 19:47:12.454329 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:12.454282    5820 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j4w9c_3443d97d-774b-4df3-abe3-3ca2d6b574b6/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:13.149620 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:13.149590    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-84cc5ff9cd-7vt6f_f3c7887d-5d2f-47a9-8808-91251d470724/manager/0.log"
Oct 30 19:47:13.613541 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:13.613504    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-dtgl9_76f5d060-4e95-48d3-99e3-815bac778442/manager/0.log"
Oct 30 19:47:14.945617 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:14.945585    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-84cc5ff9cd-7vt6f_f3c7887d-5d2f-47a9-8808-91251d470724/manager/0.log"
Oct 30 19:47:15.134719 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.134689    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-vbdd5_2452b1f3-519b-4e1d-ad93-16c233d34895/server/0.log"
Oct 30 19:47:15.429384 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.429310    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-dtgl9_76f5d060-4e95-48d3-99e3-815bac778442/manager/0.log"
Oct 30 19:47:15.485622 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.485587    5820 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm"]
Oct 30 19:47:15.489893 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.489863    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm"
Oct 30 19:47:15.527645 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.527613    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/856010e0-59f9-491b-a37f-57adc6c10db3-sys\") pod \"perf-node-gather-daemonset-qh4bm\" (UID: \"856010e0-59f9-491b-a37f-57adc6c10db3\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm"
Oct 30 19:47:15.527930 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.527910    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2ldb\" (UniqueName: \"kubernetes.io/projected/856010e0-59f9-491b-a37f-57adc6c10db3-kube-api-access-d2ldb\") pod \"perf-node-gather-daemonset-qh4bm\" (UID: \"856010e0-59f9-491b-a37f-57adc6c10db3\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm"
Oct 30 19:47:15.528104 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.528088    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/856010e0-59f9-491b-a37f-57adc6c10db3-podres\") pod \"perf-node-gather-daemonset-qh4bm\" (UID: \"856010e0-59f9-491b-a37f-57adc6c10db3\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm"
Oct 30 19:47:15.528249 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.528235    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/856010e0-59f9-491b-a37f-57adc6c10db3-lib-modules\") pod \"perf-node-gather-daemonset-qh4bm\" (UID: \"856010e0-59f9-491b-a37f-57adc6c10db3\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm"
Oct 30 19:47:15.528406 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.528391    5820 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/856010e0-59f9-491b-a37f-57adc6c10db3-proc\") pod \"perf-node-gather-daemonset-qh4bm\" (UID: \"856010e0-59f9-491b-a37f-57adc6c10db3\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm"
Oct 30 19:47:15.541640 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.541615    5820 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-v8nrk\"/\"default-dockercfg-6bfd2\""
Oct 30 19:47:15.593715 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.593677    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm"]
Oct 30 19:47:15.629304 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.629273    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2ldb\" (UniqueName: \"kubernetes.io/projected/856010e0-59f9-491b-a37f-57adc6c10db3-kube-api-access-d2ldb\") pod \"perf-node-gather-daemonset-qh4bm\" (UID: \"856010e0-59f9-491b-a37f-57adc6c10db3\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm"
Oct 30 19:47:15.629503 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.629318    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/856010e0-59f9-491b-a37f-57adc6c10db3-podres\") pod \"perf-node-gather-daemonset-qh4bm\" (UID: \"856010e0-59f9-491b-a37f-57adc6c10db3\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm"
Oct 30 19:47:15.629503 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.629336    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/856010e0-59f9-491b-a37f-57adc6c10db3-lib-modules\") pod \"perf-node-gather-daemonset-qh4bm\" (UID: \"856010e0-59f9-491b-a37f-57adc6c10db3\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm"
Oct 30 19:47:15.629503 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.629370    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/856010e0-59f9-491b-a37f-57adc6c10db3-proc\") pod \"perf-node-gather-daemonset-qh4bm\" (UID: \"856010e0-59f9-491b-a37f-57adc6c10db3\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm"
Oct 30 19:47:15.629687 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.629524    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/856010e0-59f9-491b-a37f-57adc6c10db3-podres\") pod \"perf-node-gather-daemonset-qh4bm\" (UID: \"856010e0-59f9-491b-a37f-57adc6c10db3\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm"
Oct 30 19:47:15.629687 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.629555    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/856010e0-59f9-491b-a37f-57adc6c10db3-lib-modules\") pod \"perf-node-gather-daemonset-qh4bm\" (UID: \"856010e0-59f9-491b-a37f-57adc6c10db3\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm"
Oct 30 19:47:15.629687 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.629535    5820 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/856010e0-59f9-491b-a37f-57adc6c10db3-sys\") pod \"perf-node-gather-daemonset-qh4bm\" (UID: \"856010e0-59f9-491b-a37f-57adc6c10db3\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm"
Oct 30 19:47:15.629687 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.629592    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/856010e0-59f9-491b-a37f-57adc6c10db3-sys\") pod \"perf-node-gather-daemonset-qh4bm\" (UID: \"856010e0-59f9-491b-a37f-57adc6c10db3\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm"
Oct 30 19:47:15.629687 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.629647    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/856010e0-59f9-491b-a37f-57adc6c10db3-proc\") pod \"perf-node-gather-daemonset-qh4bm\" (UID: \"856010e0-59f9-491b-a37f-57adc6c10db3\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm"
Oct 30 19:47:15.633246 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.633220    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-v8nrk\"/\"kube-root-ca.crt\""
Oct 30 19:47:15.633421 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.633276    5820 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-v8nrk\"/\"openshift-service-ca.crt\""
Oct 30 19:47:15.694665 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.694596    5820 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2ldb\" (UniqueName: \"kubernetes.io/projected/856010e0-59f9-491b-a37f-57adc6c10db3-kube-api-access-d2ldb\") pod \"perf-node-gather-daemonset-qh4bm\" (UID: \"856010e0-59f9-491b-a37f-57adc6c10db3\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm"
Oct 30 19:47:15.807125 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:15.807094    5820 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm"
Oct 30 19:47:16.218097 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:16.218059    5820 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm"]
Oct 30 19:47:16.224579 ip-10-0-133-80 kubenswrapper[5820]: W1030 19:47:16.224543    5820 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod856010e0_59f9_491b_a37f_57adc6c10db3.slice/crio-f566c82bb256c2a0f74defe75cf30ae421a1c490cc02f0055c8d3fa935b882a3 WatchSource:0}: Error finding container f566c82bb256c2a0f74defe75cf30ae421a1c490cc02f0055c8d3fa935b882a3: Status 404 returned error can't find the container with id f566c82bb256c2a0f74defe75cf30ae421a1c490cc02f0055c8d3fa935b882a3
Oct 30 19:47:16.297422 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:16.297390    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-jbdx9_48168bb8-0b54-4f25-9139-0ddc3f7950d6/manager/0.log"
Oct 30 19:47:16.505404 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:16.505333    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-mcgqw_d0e5ae11-c274-4e34-a6fe-ac8df2700247/proxy/0.log"
Oct 30 19:47:16.748013 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:16.747976    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-vbdd5_2452b1f3-519b-4e1d-ad93-16c233d34895/server/0.log"
Oct 30 19:47:17.106998 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:17.106912    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm" event={"ID":"856010e0-59f9-491b-a37f-57adc6c10db3","Type":"ContainerStarted","Data":"a51246b68fd1fdd81e6995d9984b676235d964e245374508c37897403e5ddd38"}
Oct 30 19:47:17.107178 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:17.107032    5820 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm" event={"ID":"856010e0-59f9-491b-a37f-57adc6c10db3","Type":"ContainerStarted","Data":"f566c82bb256c2a0f74defe75cf30ae421a1c490cc02f0055c8d3fa935b882a3"}
Oct 30 19:47:17.107178 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:17.107102    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm"
Oct 30 19:47:17.225538 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:17.225360    5820 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm" podStartSLOduration=2.225344726 podStartE2EDuration="2.225344726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:47:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:17.224426512 +0000 UTC m=+4357.936175210" watchObservedRunningTime="2026-10-30 19:47:17.225344726 +0000 UTC m=+4357.937093428"
Oct 30 19:47:18.128280 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:18.128225    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-jbdx9_48168bb8-0b54-4f25-9139-0ddc3f7950d6/manager/0.log"
Oct 30 19:47:18.286744 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:18.286706    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-mcgqw_d0e5ae11-c274-4e34-a6fe-ac8df2700247/proxy/0.log"
Oct 30 19:47:19.623348 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:19.623311    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-84cc5ff9cd-7vt6f_f3c7887d-5d2f-47a9-8808-91251d470724/manager/0.log"
Oct 30 19:47:20.085112 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:20.085076    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-dtgl9_76f5d060-4e95-48d3-99e3-815bac778442/manager/0.log"
Oct 30 19:47:20.160964 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:20.160921    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_50ce9dff-2437-4686-89c5-f5354c0b547e/alertmanager/0.log"
Oct 30 19:47:20.626969 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:20.626922    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_50ce9dff-2437-4686-89c5-f5354c0b547e/config-reloader/0.log"
Oct 30 19:47:21.051826 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:21.051799    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_50ce9dff-2437-4686-89c5-f5354c0b547e/kube-rbac-proxy-web/0.log"
Oct 30 19:47:21.184896 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:21.184848    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-84cc5ff9cd-7vt6f_f3c7887d-5d2f-47a9-8808-91251d470724/manager/0.log"
Oct 30 19:47:21.343225 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:21.343161    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-vbdd5_2452b1f3-519b-4e1d-ad93-16c233d34895/server/0.log"
Oct 30 19:47:21.467424 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:21.467401    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_50ce9dff-2437-4686-89c5-f5354c0b547e/kube-rbac-proxy/0.log"
Oct 30 19:47:21.687699 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:21.687600    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-dtgl9_76f5d060-4e95-48d3-99e3-815bac778442/manager/0.log"
Oct 30 19:47:21.888816 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:21.888716    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_50ce9dff-2437-4686-89c5-f5354c0b547e/kube-rbac-proxy-metric/0.log"
Oct 30 19:47:22.316584 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:22.316545    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_50ce9dff-2437-4686-89c5-f5354c0b547e/prom-label-proxy/0.log"
Oct 30 19:47:22.749285 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:22.749204    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-jbdx9_48168bb8-0b54-4f25-9139-0ddc3f7950d6/manager/0.log"
Oct 30 19:47:22.764610 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:22.764584    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_50ce9dff-2437-4686-89c5-f5354c0b547e/init-config-reloader/0.log"
Oct 30 19:47:22.903990 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:22.903942    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-mcgqw_d0e5ae11-c274-4e34-a6fe-ac8df2700247/proxy/0.log"
Oct 30 19:47:22.914383 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:22.914359    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-vbdd5_2452b1f3-519b-4e1d-ad93-16c233d34895/server/0.log"
Oct 30 19:47:23.124613 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:23.124590    5820 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-qh4bm"
Oct 30 19:47:23.618389 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:23.618356    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-4krbv_e4104cc2-a8a5-4985-83cd-e593bf12a68f/kube-state-metrics/0.log"
Oct 30 19:47:24.022556 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:24.022474    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-4krbv_e4104cc2-a8a5-4985-83cd-e593bf12a68f/kube-rbac-proxy-main/0.log"
Oct 30 19:47:24.355551 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:24.355518    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-jbdx9_48168bb8-0b54-4f25-9139-0ddc3f7950d6/manager/0.log"
Oct 30 19:47:24.438875 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:24.438842    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-4krbv_e4104cc2-a8a5-4985-83cd-e593bf12a68f/kube-rbac-proxy-self/0.log"
Oct 30 19:47:24.501136 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:24.501100    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-mcgqw_d0e5ae11-c274-4e34-a6fe-ac8df2700247/proxy/0.log"
Oct 30 19:47:24.857668 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:24.857635    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7445978df6-sf44q_cf17e11d-1428-47c8-9aea-9f6b7c831eb1/metrics-server/0.log"
Oct 30 19:47:25.274354 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:25.274284    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-cgw6j_dc7462f0-a45f-487a-99b9-3e89f783cb01/monitoring-plugin/0.log"
Oct 30 19:47:27.926611 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:27.926578    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-94chw_75209be8-a14e-40e9-9db9-d0a276c958f2/discovery/0.log"
Oct 30 19:47:28.252911 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:28.252837    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9468cdd5d-zg5gs_4de42b07-6abb-419a-99dc-89b226a2fa4e/kube-auth-proxy/0.log"
Oct 30 19:47:28.284080 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:28.284047    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x9dhg_8876fbb5-cdb6-4755-9122-8640d5c86560/node-exporter/0.log"
Oct 30 19:47:28.697483 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:28.697448    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x9dhg_8876fbb5-cdb6-4755-9122-8640d5c86560/kube-rbac-proxy/0.log"
Oct 30 19:47:28.915430 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:28.915394    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-qmbs6_3e933bb3-b9ba-4dff-841e-831b32f20537/payload-pre-processing/0.log"
Oct 30 19:47:29.115569 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:29.115523    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x9dhg_8876fbb5-cdb6-4755-9122-8640d5c86560/init-textfile/0.log"
Oct 30 19:47:29.130083 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:29.130050    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-7kxzm_88b8cb4c-656b-44e4-8632-0acff65c58ad/payload-processing/0.log"
Oct 30 19:47:29.350204 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:29.350173    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-94chw_75209be8-a14e-40e9-9db9-d0a276c958f2/discovery/0.log"
Oct 30 19:47:29.539457 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:29.539383    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-g4h9b_a9c43ecd-b447-4e07-95cf-01523d71dda9/kube-rbac-proxy-main/0.log"
Oct 30 19:47:29.681827 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:29.679903    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9468cdd5d-zg5gs_4de42b07-6abb-419a-99dc-89b226a2fa4e/kube-auth-proxy/0.log"
Oct 30 19:47:29.954506 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:29.954469    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-g4h9b_a9c43ecd-b447-4e07-95cf-01523d71dda9/kube-rbac-proxy-self/0.log"
Oct 30 19:47:30.319412 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:30.319371    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-qmbs6_3e933bb3-b9ba-4dff-841e-831b32f20537/payload-pre-processing/0.log"
Oct 30 19:47:30.375203 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:30.375176    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-g4h9b_a9c43ecd-b447-4e07-95cf-01523d71dda9/openshift-state-metrics/0.log"
Oct 30 19:47:30.534770 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:30.534738    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-7kxzm_88b8cb4c-656b-44e4-8632-0acff65c58ad/payload-processing/0.log"
Oct 30 19:47:33.762322 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:33.762283    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-bmxnr_3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84/prometheus-operator/0.log"
Oct 30 19:47:34.171844 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:34.171798    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-bmxnr_3a4a0b8a-8f26-449f-9f6e-3b91c69bfb84/kube-rbac-proxy/0.log"
Oct 30 19:47:36.333108 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:36.333079    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d4bc4b8d6-8k8jd_220393aa-93ee-4113-b373-ecd27198f170/thanos-query/0.log"
Oct 30 19:47:36.771791 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:36.771691    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d4bc4b8d6-8k8jd_220393aa-93ee-4113-b373-ecd27198f170/kube-rbac-proxy-web/0.log"
Oct 30 19:47:37.186048 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:37.186016    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d4bc4b8d6-8k8jd_220393aa-93ee-4113-b373-ecd27198f170/kube-rbac-proxy/0.log"
Oct 30 19:47:37.611655 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:37.611624    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d4bc4b8d6-8k8jd_220393aa-93ee-4113-b373-ecd27198f170/prom-label-proxy/0.log"
Oct 30 19:47:38.046886 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:38.046817    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d4bc4b8d6-8k8jd_220393aa-93ee-4113-b373-ecd27198f170/kube-rbac-proxy-rules/0.log"
Oct 30 19:47:38.207376 ip-10-0-133-80 kubenswrapper[5820]: E1030 19:47:38.207339    5820 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-klcg2" containerName="extractor"
Oct 30 19:47:38.461806 ip-10-0-133-80 kubenswrapper[5820]: I1030 19:47:38.461777    5820 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d4bc4b8d6-8k8jd_220393aa-93ee-4113-b373-ecd27198f170/kube-rbac-proxy-metrics/0.log"